Re: [virtio] Re: [PATCH v10 04/10] admin: introduce virtio admin virtqueues

[Stefan Hajnoczi <stefanha@redhat.com>]

On Mon, Mar 06, 2023 at 04:00:50PM +0800, Jason Wang wrote:
> 
> å 2023/3/6 08:03, Stefan Hajnoczi åé:
> > On Sun, Mar 05, 2023 at 04:38:59AM -0500, Michael S. Tsirkin wrote:
> > > On Fri, Mar 03, 2023 at 03:21:33PM -0500, Stefan Hajnoczi wrote:
> > > > What happens if a command takes 1 second to complete, is the device
> > > > allowed to process the next command from the virtqueue during this time,
> > > > possibly completing it before the first command?
> > > > 
> > > > This requires additional clarification in the spec because "they are
> > > > processed by the device in the order in which they are queued" does not
> > > > explain whether commands block the virtqueue (in order completion) or
> > > > not (out of order completion).
> > > Oh I begin to see. Hmm how does e.g. virtio scsi handle this?
> > virtio-scsi, virtio-blk, and NVMe requests may complete out of order.
> > Several may be processed by the device at the same time.
> > 
> > They rely on multi-queue for abort operations:
> > 
> > In virtio-scsi the abort requests (VIRTIO_SCSI_T_TMF_ABORT_TASK) are
> > sent on the control virtqueue. The the request identifier namespace is
> > shared across all virtqueues so it's possible to abort a request that
> > was submitted to any command virtqueue.
> > 
> > NVMe also follows the same design where abort commands are sent on the
> > Admin Submission Queue instead of an I/O Submission Queue. It's possible
> > to identify NVMe requests by <Submission Queue ID, Command Identifier>.
> > 
> > virtio-blk doesn't support aborting requests.
> > 
> > I think the logic behind this design is that if a queue gets stuck
> > processing long-running requests, then the device should not be forced
> > to perform lookahead in the queue to find abort commands. A separate
> > control/admin queue is used for the abort requests.
> 
> 
> Or device need mandate some kind of QOS here, e.g a request must be complete
> in some time. Otherwise we don't have sufficient reliability for using it as
> management task?

Yes, if all commands can be executed in bounded time then a guarantee is
possible.

Here is an example where that's hard: imagine a virtio-blk device backed
by network storage. When an admin queue command is used to delete a
group member, any of the group member's in-flight I/O requests need to
be aborted. If the network hangs while the group member is being
deleted, then the device can't complete an orderly shutdown of I/O
requests in a reasonable time.

That example shows a basic group admin command that I think Michael is
about to propose. We can't avoid this problem by not making it a group
admin command - it needs to be a group admin command. So I think it's
likely that there will be admin commands that take an unbounded amount
of time to complete. One way to achieve what you mentioned is timeouts.

Stefan

Attachment: signature.asc
Description: PGP signature