Re: [virtio-comment] [PATCH RFC] virtio: introduce VIRTIO_F_DEVICE_STOP

[Jason Wang <jasowang@redhat.com>]

On 2020/12/28 äå2:21, Halil Pasic wrote:
> On Sun, 27 Dec 2020 05:00:05 -0500
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
> > On Fri, Dec 25, 2020 at 08:38:35AM +0100, Halil Pasic wrote:
> > > When driver is trying to set DEVICE_STOPPED, the device MUST not
> > > process new avail requests and MUST complete all requests that is
> > > currently processing before setting DEVICE_STOPPED.
> > ...
> >
> > > Since we have a synchronous API setting DEVICE_STOPPED would also have to
> > > block until all in-flight requests are completed.
> > Judging from the surrounding discussion,
> > when you say complete you seem to mean "use", and
> My intention was merely to paraphrase Jason's proposal which says:
>
> +When driver is trying to set DEVICE_STOPPED, the device MUST not
> +process new avail requests and MUST complete all requests that is
> +currently processing before setting DEVICE_STOPPED.
>
> > I'm not sure how you define in flight, but
> > it seems there could be many of these (up to a full queue?)
> In the follow-on discussion 'in-flight' emerged as an alternative to
> 'all requests that is currently processing' form the proposed text.
>
> A large part of the discussion is, IMHO, about finding precise
> definitions, for what the quoted paragraph is trying to express.
>
> > so waiting for all available buffers to be
> > used might indeed require an asynchronous interface,
> > which gets complex very quickly.


Some part of the virtio has enforced an asynchronous interface during reset:

For MMIO the spec said:

"""

To stop using the queue the driver MUST write zero (0x0) to this 
QueueReady and MUST read the value back to ensure synchronization.

"""

For PCI it said:

"""

After writing 0 to device_status, the driver MUST wait for a read of 
device_status to return 0 before reinitializing the device.

"""


> > However wouldn't it be possible for device to just cancel
> > processing available buffers even if it started processing
> > them? Some buffers could be in indeterminate state
> > (e.g. we might not have a way to know how much data did
> > device have time to write into a buffer).
> Maybe Jason can answer this one.


For networking device, it should be possible (packet could be dropped or 
duplicated). But I'm not sure it works for other device. E.g for the 
block devices that needs to communicate with a remote backend. Waat's 
more important, my understadning is that the intermediate state is 
something that we need to avoid (hard to be migrated anyhow).


> > Making it clearer what does "complete" mean here might help.
> I think what we are trying to accomplish here, is avoiding, having
> non-standardised state in device (that can not be dropped) when
> migrating.
>
> I'm still struggling with wrapping my mind around the problem. AFAIU
> migration and migratability is not really a feature of the virtio
> standard, but can be a feature of it's implementation
> (e.g. QEMU & KVM), where the standard does help a great deal by having
> certain aspects of the operation and interaction nailed down.


It looks like a must (at least from the level of semantics) for 
designing software API for vDPA. Using a standard spec may help to avoid 
subtle misunderstanding of different vendors.

Thanks


> Regards,
> Halil