Re: [virtio-comment] [RFC PATCH 0/5] virtio: introduce SUSPEND bit and vq state

["Zhu, Lingshan" <lingshan.zhu@intel.com>]

On 8/15/2023 9:38 AM, Jason Wang wrote:
> On Tue, Aug 15, 2023 at 12:28âAM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > On Tue, Aug 15, 2023 at 03:28:59AM +0800, Zhu Lingshan wrote:
> > > This seires introduces
> > > 1)a new SUSPEND bit in the device status
> > > Which is used to suspend the device, so that the device states
> > > and virtqueue states are stablized.
> > >
> > > 2)virtqueue state and accessors, to get and set last_avail_idx
> > > and last_used_idx of virtqueues.
> > >
> > > The main usecase of these new facilities is Live Migration.
> > >
> > > Furture work: dirty page tracking and in-flight descriptors.
> > >
> > > This is RFC, this series carries on Jason and Eugenio's pervious work.
> > >
> > > Any comments are welcome.
> > In order to support stateful devices like virtiofs, virtio-gpu, or
> > virtio-crypto it would be necessary to add device state save/load
> > functionality too.
> +1 and before save/load, we need to define those states first.
>
> E.g there are patches that try to fix the GPU suspend/hibernation
> which probably requires those facilities as well.
I agree, we should define the states of the device types first, like
virtio-fs. For virtio-net, one of the states if in-flight descriptors,
and this is planned work.

> > Even virtio-net needs device state save/load functionality if the VMM is
> > not intercepting the stateful controlq.
> Yes, this could be done via trap and emulation.
I agree
> > In-flight descriptors can be included in the device state.
> Probably.
Yes, in-flight descriptors and dirty page tracking are in the plan. This 
series only
implements SUSPEND and vq state accessors as the first RFC try to 
collect comments.
> > In fact, the only reason why I think VIRTIO_F_QUEUE_STATE makes sense is
> > for easy integration into existing vhost software stacks. Otherwise we
> > should just define a device state save/load interface along with
> > standard device state serialization for devices where migration
> > interoperability is possible. VIRTIO_F_QUEUE_STATE is a small subset of
> > device state save/load.
> For simple devices like virtio-net, queue state should be sufficient
> for hardware.
>
> > I think the convenience of integrating into existing vhost software
> > stacks is worth the duplication, but I wanted to mention that this
> > interface will not be enough to live migrate all device types and we'll
> > need something more in the future.
> Yes, the patch is a start for simple stateless devices.
Yes!

Thanks!
> Thanks
>
> > Stefan
> >
> > > Zhu Lingshan (5):
> > >    virtio: introduce SUSPEND bit in device status
> > >    virtio: introduce vq state as basic facility
> > >    virtio: The actions by the device upon SUSPEND
> > >    virtqueue: constraints for virtqueue state
> > >    virtio-pci: implement VIRTIO_F_QUEUE_STATE
> > >
> > >   content.tex       | 120 ++++++++++++++++++++++++++++++++++++++++++++++
> > >   transport-pci.tex |  15 ++++++
> > >   2 files changed, 135 insertions(+)
> > >
> > > --
> > > 2.35.3
> > >
> > >
> > > This publicly archived list offers a means to provide input to the
> > > OASIS Virtual I/O Device (VIRTIO) TC.
> > >
> > > In order to verify user consent to the Feedback License terms and
> > > to minimize spam in the list archive, subscription is required
> > > before posting.
> > >
> > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > > List help: virtio-comment-help@lists.oasis-open.org
> > > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > > Committee: https://www.oasis-open.org/committees/virtio/
> > > Join OASIS: https://www.oasis-open.org/join/