Re: [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices

[Stefan Hajnoczi <stefanha@redhat.com>]

On Fri, Sep 08, 2023 at 01:03:26PM +0100, Alex Bennée wrote:
> 
> Stefan Hajnoczi <stefanha@redhat.com> writes:
> 
> > On Fri, Sep 01, 2023 at 12:00:18PM +0100, Alex Bennée wrote:
> >> Currently QEMU has to know some details about the VirtIO device
> >> supported by a vhost-user daemon to be able to setup the guest. This
> >> makes it hard for QEMU to add support for additional vhost-user
> >> daemons without adding specific stubs for each additional VirtIO
> >> device.
> >> 
> >> This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE)
> >> which the back-end can advertise which allows a probe message to be
> >> sent to get all the details QEMU needs to know in one message.
> >> 
> >> Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and
> >> VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user
> >> daemons which are capable of handling all aspects of the VirtIO
> >> transactions with only a generic stub on the QEMU side. These daemons
> >> can also be used without QEMU in situations where there isn't a full
> >> VMM managing their setup.
> >> 
> >> Signed-off-by: Alex Bennée <alex.bennee@linaro.org>
> >
> > I think the mindset for this change should be "vhost-user is becoming a
> > VIRTIO Transport". VIRTIO Transports have a reasonably well-defined
> > feature set in the VIRTIO specification. The goal should be to cover
> > every VIRTIO Transport operation via vhost-user protocol messages so
> > that the VIRTIO device model can be fully conveyed over vhost-user.
> 
> Is it though? The transport is a guest visible construct whereas
> vhost-user is purely a backend implementation detail that should be
> invisible to the guest.

No, the transport is not necessarily guest-visible. The vhost-user model
is that the front-end emulates a VIRTIO device and some aspects of that
device are delegated to the vhost-user back-end.

In other words, the vhost-user device is not the same as the VIRTIO
device that the guest sees, but it's still important for the vhost-user
back-end to be a VIRTIO Transport because that's how we can be sure it
supports the VIRTIO device model properly.

> 
> Also the various backends do things a different set of ways. The
> differences between MMIO and PCI are mostly around where config space is
> and how IRQs are handled. For CCW we do actually have a set of commands
> we can look at:
> 
>   #define CCW_CMD_SET_VQ 0x13 
>   #define CCW_CMD_VDEV_RESET 0x33 
>   #define CCW_CMD_SET_IND 0x43 
>   #define CCW_CMD_SET_CONF_IND 0x53 
>   #define CCW_CMD_SET_IND_ADAPTER 0x73 
>   #define CCW_CMD_READ_FEAT 0x12 
>   #define CCW_CMD_WRITE_FEAT 0x11 
>   #define CCW_CMD_READ_CONF 0x22 
>   #define CCW_CMD_WRITE_CONF 0x21 
>   #define CCW_CMD_WRITE_STATUS 0x31 
>   #define CCW_CMD_READ_VQ_CONF 0x32 
>   #define CCW_CMD_SET_VIRTIO_REV 0x83 
>   #define CCW_CMD_READ_STATUS 0x72
> 
> which I think we already have mappings for.

Yes, there are differences between the transports. vhost-user uses
eventfds (callfd/kickfd) instead of interrupts.

> > Anything less is yet another ad-hoc protocol extension that will lead to
> > more bugs and hacks when it turns out some VIRTIO devices cannot be
> > expressed due to limitations in the protocol.
> 
> I agree we want to do this right.
> 
> > This requires going through the VIRTIO spec to find a correspondence
> > between virtio-pci/virtio-mmio/virtio-ccw's interfaces and vhost-user
> > protocol messages. In most cases vhost-user already offers messages and
> > your patch adds more of what is missing. I think this effort is already
> > very close but missing the final check that it really matches the VIRTIO
> > spec.
> >
> > Please do the comparison against the VIRTIO Transports and then adjust
> > this patch to make it clear that the back-end is becoming a full-fledged
> > VIRTIO Transport:
> > - The name of the patch series should reflect that.
> > - The vhost-user protocol feature should be named F_TRANSPORT.
> > - The messages added in this patch should have a 1:1 correspondence with
> >   the VIRTIO spec including using the same terminology for consistency.
> >
> > Sorry for the hassle, but I think this is a really crucial point where
> > we have the chance to make vhost-user work smoothly in the future...but
> > only if we can faithfully expose VIRTIO Transport semantics.
> 
> I wonder if first be handled by cleaning up the VirtIO spec to make it
> clear what capabilities each transport needs to support?

It's a fair point that the VIRTIO spec does not provide an interface
definition for the VIRTIO Transport or at least a definitive list of
requirements. The requirements are implicit (i.e. it is assumed that
very transport provides a way to set the virtqueue descriptor table
addresses) so it's necessary to review the existing transports to
understand their functionality.

If you want to create a list of the requirements for a VIRTIO Transport
and propose a patch to the VIRTIO spec then that would be great, but I
don't think that stops this patch series. It's possible to review
virtio-pci/virtio-mmio/virtio-ccw and check that there is equivalent
functionality in the vhost-user protocol.

Stefan

Attachment: signature.asc
Description: PGP signature