Re: [virtio-comment] [PATCH RFC] virtio: introduce VIRTIO_F_DEVICE_STOP

[Jason Wang <jasowang@redhat.com>]

----- Original Message -----
> On Mon, Dec 28, 2020 at 03:05:43PM +0800, Jason Wang wrote:
> > 
> > On 2020/12/27 äå7:12, Michael S. Tsirkin wrote:
> > > On Fri, Dec 25, 2020 at 02:45:28PM +0800, Jason Wang wrote:
> > > > > I tend to say, that from a perspective of the driver, all requests
> > > > > that
> > > > > are available, and not yet used, are in-flight. So we have to be very
> > > > > careful when wording this requirement, to avoid misunderstandings. I
> > > > > don't think the first RFC is good enough. I will think some more
> > > > > about
> > > > > this.
> > > > 
> > > > Yes, I agree. The problem is that the spec doesn't describe how device
> > > > work,
> > > > so if we want to be more accurate, it might require some work not only
> > > > for
> > > > stop but also for e.g reset (something like in flight has been used by
> > > > the
> > > > spec in that case).
> > > You probably mean the DEVICE_NEEDS_RESET description, right?
> > > 
> > > 	For example, the driver can't assume requests in flight will be
> > > 	completed if DEVICE_NEEDS_RESET is set, nor can it assume that
> > > 	they have not been completed.  A good implementation will try to
> > > 	recover by issuing a reset.
> > > 
> > > yes, DEVICE_NEEDS_RESET is unfortunately underspecified which likely
> > > is related to the fact it is not widely implemented.
> > 
> > 
> > For both NEEDS_RESET and device reset.
> > 
> > For NEEDS_RESET, we use "in flight" and "completed" without an actual
> > definition.
> > 
> > For device reset, we don't even define what is the device expected (e.g how
> > are "in flight" requests expected to be processed) to behave.
> > 
> > Thanks
> 
> Because device is expected to just stop:
> 
> 	None of the virtqueues
> 	of a device are live once the device has been reset.

So it's something similar to what DEVICE_STOPPED wants.

> 
> and it's driver's job to recover buffers:
> 
> 	Thus a driver MUST ensure a virtqueue isn't live (by device reset) before
> 	removing exposed buffers.
> 
> what happened to buffers which were not used?
> 
> I think it's a quality of implementation/device specific issue, e.g. for net:
> for transmit, if device sends a packet without using the buffer,

This sounds like an intermediate state we need to avoid in the
migration. Can we mandate the device to make the buffer used in this
case? For networking device, it should be not slow, just wait for the
DMA to be completed is sufficient.

> then driver will resend the packet leading to duplicates.
> for receive, it's a packet drop, slightly less of a problem.

For net yes.

> 
> 
> my question is whether such behaviour is sufficient for migration?

I suspect it's not, we need to either

1) wait for the buffer to be used

or 

2) fail the migration if the device is not stopped in a short time

> if not can we really describe it generally?

It looks like a virtqueue level issue, so we need to try.

> it's possible we need to
> describe it per device type.

Maybe.

Thanks

> 
> --
> MST
> 
> 
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
> 
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
> 
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
> 
>