[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] [RFC PATCH v2] docs/interop: define PROBE feature for vhost-user VirtIO devices
Albert Esteve <aesteve@redhat.com> writes:
> This looks great! Thanks for this proposal.
>
> On Fri, Sep 1, 2023 at 1:00âPM Alex BennÃe <alex.bennee@linaro.org> wrote:
>
>Â Currently QEMU has to know some details about the VirtIO device
>Â supported by a vhost-user daemon to be able to setup the guest. This
>Â makes it hard for QEMU to add support for additional vhost-user
>Â daemons without adding specific stubs for each additional VirtIO
>Â device.
>
>Â This patch suggests a new feature flag (VHOST_USER_PROTOCOL_F_PROBE)
>Â which the back-end can advertise which allows a probe message to be
>Â sent to get all the details QEMU needs to know in one message.
>
>Â Together with the existing features VHOST_USER_PROTOCOL_F_STATUS and
>Â VHOST_USER_PROTOCOL_F_CONFIG we can create "standalone" vhost-user
>Â daemons which are capable of handling all aspects of the VirtIO
>Â transactions with only a generic stub on the QEMU side. These daemons
>Â can also be used without QEMU in situations where there isn't a full
>Â VMM managing their setup.
>
>Â Signed-off-by: Alex BennÃe <alex.bennee@linaro.org>
>
>Â ---
>Â v2
>Â Â - dropped F_STANDALONE in favour of F_PROBE
>Â Â - split probe details across several messages
>Â Â - probe messages don't automatically imply a standalone daemon
>Â Â - add wording where probe details interact (F_MQ/F_CONFIG)
>Â Â - define VMM and make clear QEMU is only one of many potential VMMs
>Â Â - reword commit message
>Â ---
>Â Âdocs/interop/vhost-user.rst | 90 ++++++++++++++++++++++++++++++++-----
> Âhw/virtio/vhost-user.c   | 8 ++++
>Â Â2 files changed, 88 insertions(+), 10 deletions(-)
>
>Â diff --git a/docs/interop/vhost-user.rst b/docs/interop/vhost-user.rst
>Â index 5a070adbc1..ba3b5e07b7 100644
>Â --- a/docs/interop/vhost-user.rst
>Â +++ b/docs/interop/vhost-user.rst
>Â @@ -7,6 +7,7 @@ Vhost-user Protocol
>Â Â..
>Â Â ÂCopyright 2014 Virtual Open Systems Sarl.
>Â Â ÂCopyright 2019 Intel Corporation
>Â +Â Copyright 2023 Linaro Ltd
>Â Â ÂLicence: This work is licensed under the terms of the GNU GPL,
>Â Â Â Â Â Â Â version 2 or later. See the COPYING file in the top-level
>Â Â Â Â Â Â Â directory.
>Â @@ -27,17 +28,31 @@ The protocol defines 2 sides of the communication, *front-end* and
>Â Â*back-end*. The *front-end* is the application that shares its virtqueues, in
>Â Âour case QEMU. The *back-end* is the consumer of the virtqueues.
>
>Â -In the current implementation QEMU is the *front-end*, and the *back-end*
>Â -is the external process consuming the virtio queues, for example a
>Â -software Ethernet switch running in user space, such as Snabbswitch,
>Â -or a block device back-end processing read & write to a virtual
>Â -disk. In order to facilitate interoperability between various back-end
>Â -implementations, it is recommended to follow the :ref:`Backend program
>Â -conventions <backend_conventions>`.
>Â +In the current implementation a Virtual Machine Manager (VMM) such as
>Â +QEMU is the *front-end*, and the *back-end* is the external process
>Â +consuming the virtio queues, for example a software Ethernet switch
>Â +running in user space, such as Snabbswitch, or a block device back-end
>Â +processing read & write to a virtual disk. In order to facilitate
>Â +interoperability between various back-end implementations, it is
>Â +recommended to follow the :ref:`Backend program conventions
>Â +<backend_conventions>`.
>
>Â ÂThe *front-end* and *back-end* can be either a client (i.e. connecting) or
>Â Âserver (listening) in the socket communication.
>
>Â +Probing device details
>Â +----------------------
>Â +
>Â +Traditionally the vhost-user daemon *back-end* shares configuration
>Â +responsibilities with the VMM *front-end* which needs to know certain
>Â +key bits of information about the device. This means the VMM needs to
>Â +define at least a minimal stub for each VirtIO device it wants to
>Â +support. If the daemon supports the right set of protocol features the
>Â +VMM can probe the daemon for the information it needs to setup the
>Â +device. See :ref:`Probing features for standalone daemons
>Â +<probing_features>` for more details.
>Â +
>Â +
>Â ÂSupport for platforms other than Linux
>Â Â--------------------------------------
>
>Â @@ -316,6 +331,7 @@ replies. Here is a list of the ones that do:
>Â Â* ``VHOST_USER_GET_VRING_BASE``
>Â Â* ``VHOST_USER_SET_LOG_BASE`` (if ``VHOST_USER_PROTOCOL_F_LOG_SHMFD``)
>Â Â* ``VHOST_USER_GET_INFLIGHT_FD`` (if ``VHOST_USER_PROTOCOL_F_INFLIGHT_SHMFD``)
>Â +* ``VHOST_USER_GET_BACKEND_SPECS`` (if ``VHOST_USER_PROTOCOL_F_STANDALONE``)
>
>Â Â.. seealso::
>
>Â @@ -396,9 +412,10 @@ must support changing some configuration aspects on the fly.
>Â ÂMultiple queue support
>Â Â----------------------
>
> -Many devices have a fixed number of virtqueues. In this case the front-end
>Â -already knows the number of available virtqueues without communicating with the
>Â -back-end.
>Â +Many devices have a fixed number of virtqueues. In this case the
>Â +*front-end* usually already knows the number of available virtqueues
>Â +without communicating with the back-end. For standalone daemons this
>Â +number can be can be probed with the ``VHOST_USER_GET_MIN_VQ`` message.
>
> ÂSome devices do not have a fixed number of virtqueues. Instead the maximum
> Ânumber of virtqueues is chosen by the back-end. The number can depend on host
>Â @@ -885,6 +902,23 @@ Protocol features
>Â Â Â#define VHOST_USER_PROTOCOL_F_CONFIGURE_MEM_SLOTSÂ 15
>Â Â Â#define VHOST_USER_PROTOCOL_F_STATUSÂ Â Â Â Â Â Â Â16
>Â Â Â#define VHOST_USER_PROTOCOL_F_XEN_MMAPÂ Â Â Â Â Â Â17
>Â +Â #define VHOST_USER_PROTOCOL_F_PROBEÂ Â Â Â Â Â Â Â 18
>Â +
>Â +.. _probing_features:
>Â +
>Â +Probing features for standalone daemons
>Â +---------------------------------------
>Â +
>Â +The protocol feature ``VHOST_USER_PROTOCOL_F_PROBE`` enables a number
>Â +of additional messages which allow the *front-end* to probe details
>Â +about the VirtIO device from the *back-end*. However for a *back-end*
>Â +to be described as standalone it must also support:
>Â +
>Â +Â * ``VHOST_USER_PROTOCOL_F_STATUS``
>Â +Â * ``VHOST_USER_PROTOCOL_F_CONFIG`` (if there is a config space)
>Â +
>Â +which are required to ensure the *back-end* daemon can operate
>Â +without the *front-end* managing some aspects of its configuration.
>
>Â ÂFront-end message types
>Â Â-----------------------
>Â @@ -1440,6 +1474,42 @@ Front-end message types
>Â Â Âquery the back-end for its device status as defined in the Virtio
>Â Â Âspecification.
>
>Â +``VHOST_USER_GET_DEVICE_ID``
>Â +Â :id: 41
>Â +Â :request payload: N/A
>Â +Â :reply payload: ``u32``
>Â +
>Â +Â When the ``VHOST_USER_PROTOCOL_F_PROBE`` protocol feature has been
>Â +Â successfully negotiated, this message is submitted by the front-end
>Â +Â to query what VirtIO device the back-end support. This is intended
>Â +Â to remove the need for the front-end to know ahead of time what the
>Â +Â VirtIO device the backend emulates is.
>Â +
>Â +``VHOST_USER_GET_CONFIG_SIZE``
>Â +Â :id: 42
>Â +Â :request payload: N/A
>Â +Â :reply payload: ``u32``
>Â +
>Â +Â When the ``VHOST_USER_PROTOCOL_F_PROBE`` protocol feature has been
>Â +Â successfully negotiated, this message is submitted by the front-end
>Â +Â to query the size of the VirtIO device's config space. This is
>Â +Â intended to remove the need for the front-end to know ahead of time
>Â +Â what the size is. Replying with 0 when
>Â +Â ``VHOST_USER_PROTOCOL_F_CONFIG`` has been negotiated would indicate
>Â +Â an bug.
>Â +
>Â +``VHOST_USER_GET_MIN_VQ``
>Â +Â :id: 43
>Â +Â :request payload: N/A
>Â +Â :reply payload: ``u32``
>Â +
>Â +Â When the ``VHOST_USER_PROTOCOL_F_PROBE`` protocol feature has been
>Â +Â successfully negotiated, this message is submitted by the front-end to
>Â +Â query minimum number of VQ's required to support the device. A
>Â +Â device may support more than this number of VQ's if it advertises
>Â +Â the ``VHOST_USER_PROTOCOL_F_MQ`` protocol feature. Reporting a
>Â +Â number greater than the result of ``VHOST_USER_GET_QUEUE_NUM`` would
>Â +Â indicate a bug.
>
> Maybe I lack some background, but not sure what min_vq is here?
There will be a minimum number of queues you need to support the device.
For example the virtio-sound spec specifies you need four queues:
control, event, tx, rx
> This looks like quering the number of VQs the backend requires/uses.
> Which, in case of MQ, it may be bigger (which is where I assume comes the `min`
> part, if we consider `VHOST_USER_GET_QUEUE_NUM` the `max`).
The MQ extension is currently used by networking but in theory any
device could attempt to parallelism by extending the number of virt
queues needed. So for net you get:
 receiveq1
 transmitq1
 optional controlq
So VHOST_USER_GET_MIN_VQ would report 2 or 3 (if VIRTIO_NET_F_CTRL_VQ is
negotiated). However VHOST_USER_GET_QUEUE_NUM is only usable if
VIRTIO_NET_F_MQ has been negotiated and could report more.
>
> Couldn't we reuse the `VHOST_USER_GET_QUEUE_NUM` type for this?
>Â
>Â ÂBack-end message types
>Â Â----------------------
>Â diff --git a/hw/virtio/vhost-user.c b/hw/virtio/vhost-user.c
>Â index 8dcf049d42..4d433cdf2b 100644
>Â --- a/hw/virtio/vhost-user.c
>Â +++ b/hw/virtio/vhost-user.c
>Â @@ -202,6 +202,13 @@ typedef struct VhostUserInflight {
>Â Â Â Âuint16_t queue_size;
>Â Â} VhostUserInflight;
>
>Â +typedef struct VhostUserBackendSpecs {
>Â +Â Â uint32_t device_id;
>Â +Â Â uint32_t config_size;
>Â +Â Â uint32_t min_vqs;
>Â +Â Â uint32_t max_vqs;
>Â +} VhostUserBackendSpecs;
>Â +
>Â Âtypedef struct {
>Â Â Â ÂVhostUserRequest request;
>
>Â @@ -226,6 +233,7 @@ typedef union {
>Â Â Â Â Â ÂVhostUserCryptoSession session;
>Â Â Â Â Â ÂVhostUserVringArea area;
>Â Â Â Â Â ÂVhostUserInflight inflight;
>Â +Â Â Â Â VhostUserBackendSpecs specs;
Oops these snuck in, I shall clean them up
>Â Â} VhostUserPayload;
>
>Â Âtypedef struct VhostUserMsg {
>Â --
>Â 2.39.2
>
>Â ---------------------------------------------------------------------
>Â To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
>Â For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
--
Alex BennÃe
Virtualisation Tech Lead @ Linaro
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]