[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-comment] [PATCH RFC] virtio: introduce VIRTIO_F_DEVICE_STOP
On 2020/12/24 äå12:52, Halil Pasic wrote:
On Tue, 22 Dec 2020 20:51:04 +0800 Jason Wang <email@example.com> wrote:From Qemu point of view, vhost-vDPA is just another type of vhost backend. Qemu needs to stop virtio (vhost) before it can do migration. So we require vDPA devices to have the ability of stopping or pausing its datapath. If the vDPA device is by chance the virtio-PCI device, it needs an interface for receiving stop/resume command from the driver. So the devce stop/resume command was sent from Qemu to vhost-VDPA, then to vDPA parent which could be a virtio-PCI device in this case.But QEMU implements the _device_, not the driver, doesn't it?The device is implemented with the co-operation between Qemu and vhost-vDPA. During migration qemu need to stop the virtio-net device, then vhost must be stopped as well.Sorry for the lazy/stupid question (it's been a while since I last understood that code), how does this work with vhost based virtio-net? I don't see ioctl that stops the old classic vhost (no vdpa). I have a fuzzy remembrance that we 'stop' the notifiers (ioeventfd & irqfd) but I'm not sure.
Each vhost devices implements its own ioctl:For vhost-net, it's VHOST_NET_SET_BACKEND, passing -1 as socket fd will stop the device.
For vhost-vsock, it's VHOST_VSOCK_SET_RUNNING. I don't check SCSI but it should have something similar.
And since we are at virtio-net,
The feature is not net specific.
I ask myself how are 'new requests' and 'requests that is (the device) currently processing' defined for the receive functionality and the rx queue(s). Remember the latter ('in flight requests') need to be all completed. I mean 'in-flight' is pretty straight-forward for something like virtio-blk, or even for tx but I have a hard time with rx.
I think it should be no difference from the view of virtqueue. It's the requests that have been read from avail ring but not added to the used ring.
Actually this ties to the visibility of virtqueue internal state. Usually device maintains a tail pointer (last_avail_idx).Â So in the simplest case when no indices wrap and no out-of-order, requests belongs to [last_avail_idx, used_idx) are considered as in-flight.
And IIUC, vhost-VDPA and the vDPA parent are also on the device side. I feel like I'm missing something essential here.Virtio-PCI driver could be a vDPA parent as well in this case. So we need stop the virtio-pci device.I used to think about vdpa like a vehicle to make partial virtio support in HW viable and easy. I.e. when I hear vdpa I have something like this in mind: https://www.redhat.com/cms/managed-files/2019-10-02-vdpa-figure5.jpg Of course the 'physical nic' from the linked picture can be a nic that supports both the virtio data plane and the virtio control plane (i.e. what you are referring to as a virtio-pci device). But do we still expect that device to be connected via vdpa?
Why not? With the help of vhost-vDPA, we get the wire speed and live migration support for virtio-pci device.
The second question is not strictly on topic, but I'm still curious, what do we plan to do with a lower device that does not support virtio control plane?
Yes, the vDPA device just need to implement the same semantic not the same interface. You may refer mlx5e vDPA driver in the kernel source.
In that case we can't go via DEVICE_STOP. Does that mean, there has to be a vendor specific mechanism equivalent to the DEVICE_STOP mechanism, or otherwise we are non-migratable?
Exactly. Thanks  https://lwn.net/Articles/838978/