Re: [RFC] virtio-net: support access and control the member devices

["Michael S. Tsirkin" <mst@redhat.com>]

On Thu, Aug 03, 2023 at 04:31:50PM +0800, Xuan Zhuo wrote:
> Now we have the concept of the device group. This allows us to
> support sr-iov and siov more conveniently.
> 
> The following description uses pf, vf as an example. The scene with sf
> should be similar.
> 
> According to my understanding, the management at the level of virtqueue
> should be handled by admin queue. And the net related management is
> still managed by cvq of pf. So this patch works with the cvq of the pf.
> 
> The problem I am trying to solve is when there are multiple vf
> virtio-net devices under one pf.
> It is necessary for administrators to query the status and information
> of each vf and configure mac.
> 
> Signed-off-by: Xuan Zhuo <xuanzhuo@linux.alibaba.com>


I think we should use admin queue for this, and not cvq.

> ---
>  device-types/net/description.tex        | 72 +++++++++++++++++++++++--
>  device-types/net/device-conformance.tex |  1 +
>  2 files changed, 70 insertions(+), 3 deletions(-)
> 
> diff --git a/device-types/net/description.tex b/device-types/net/description.tex
> index 76585b0..1494a18 100644
> --- a/device-types/net/description.tex
> +++ b/device-types/net/description.tex
> @@ -88,6 +88,9 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
>  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
>      channel.
>  
> +\item[VIRTIO_NET_F_ACCESS_MEMBER_DEVICE(51)] Device can access and control the
> +    member devices.
> +
>  \item[VIRTIO_NET_F_HASH_TUNNEL(51)] Device supports inner header hash for encapsulated packets.
>  
>  \item[VIRTIO_NET_F_VQ_NOTF_COAL(52)] Device supports virtqueue notification coalescing.
> @@ -1156,6 +1159,7 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>          u8 command;
>          u8 command-specific-data[];
>          u8 ack;
> +        u8 command-specific-data-reply[];
>  };
>  
>  /* ack values */
> @@ -1164,9 +1168,11 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  \end{lstlisting}
>  
>  The \field{class}, \field{command} and command-specific-data are set by the
> -driver, and the device sets the \field{ack} byte. There is little it can
> -do except issue a diagnostic if \field{ack} is not
> -VIRTIO_NET_OK.
> +driver, and the device sets the \field{ack} byte and optionally
> +\field{command-specific-data-reply}. There is little the driver can
> +do except issue a diagnostic if \field{ack} is not VIRTIO_NET_OK.
> +
> +The command VIRTIO_NET_CTRL_MEMBER_GET_INFO contains \field{command-specific-data-reply}.
>  
>  \paragraph{Packet Receive Filtering}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Packet Receive Filtering}
>  \label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Setting Promiscuous Mode}%old label for latexdiff
> @@ -1805,6 +1811,66 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi
>  
>  Upon reset, a device MUST initialize all coalescing parameters to 0.
>  
> +\paragraph{Member Device}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Member Info}
> +
> +If a deivce is the own deivce inside one device group and the feature
> +VIRTIO_NET_F_ACCESS_MEMBER_DEVICE is negotiated, the own driver can obtain
> +the info and change the configuration of the member devices of this device
> +group.
> +
> +\begin{lstlisting}
> +struct virtio_net_ctrl_member {
> +        le64 group_member_id;
> +};
> +
> +struct virtio_net_ctrl_member_info_reply {
> +        u8 mac[6];
> +        le16 status;
> +        le16 max_virtqueue_pairs;
> +        le16 mtu;
> +        le32 speed;
> +        u8 duplex;
> +        u8 driver_ok;
> +        le16 current_virtqueue_pairs;
> +};
> +
> +struct virtio_net_ctrl_member_set_mac {
> +        le64 group_member_id;
> +        u8 mac[6];
> +};
> +
> +#define VIRTIO_NET_CTRL_MEMBER 7
> + #define VIRTIO_NET_CTRL_MEMBER_GET_INFO  0
> + #define VIRTIO_NET_CTRL_MEMBER_SET_MAC   1
> +\end{lstlisting}
> +
> +\devicenormative{\subparagraph}{Member Device}{Device Types / Network Device / Device Operation / Control Virtqueue / Member Device}
> +
> +The device only offers the feature VIRTIO_NET_F_ACCESS_MEMBER_DEVICE when it is
> +the own device.
> +
> +If the \field{group_member_id} is not within the range of the member device id,
> +the own device MUST respond with VIRTIO_NET_ERR.
> +
> +struct virtio_net_ctrl_member_info_reply is used to reply the command
> +VIRTIO_NET_CTRL_MEMBER_GET_INFO via \field{command-specific-data-reply}.
> +
> +If the member device specified by the \field{group_member_id} is in the
> +DRIVER_OK status, the own device MUST set the \field{driver_ok} to 1 and set the
> +\field{current_virtqueue_pairs} to the number of virtqueue pairs that are being
> +used by the member device. Otherwise, both \field{driver_ok} and
> +\field{current_virtqueue_pairs} MUST be set to zero.
> +
> +All other fields of the struct virtio_net_ctrl_member_info_reply MUST be equal
> +to the values in the struct virtio_net_config of the member device.
> +
> +If the member device specified by the \field{group_member_id} supports
> +VIRTIO_NET_F_MAC and its status is not DRIVER_OK, the own device MUST reply to
> +VIRTIO_NET_CTRL_MEMBER_SET_MAC with VIRTIO_NET_OK, and the member device MUST
> +offer the \field{mac} to the member driver via the struct virtio_net_config.
> +Otherwise, the own device MUST reply to the command
> +VIRTIO_NET_CTRL_MEMBER_SET_MAC with VIRTIO_NET_ERR.
> +
>  \subsubsection{Legacy Interface: Framing Requirements}\label{sec:Device
>  Types / Network Device / Legacy Interface: Framing Requirements}
>  
> diff --git a/device-types/net/device-conformance.tex b/device-types/net/device-conformance.tex
> index f88f48b..9af3293 100644
> --- a/device-types/net/device-conformance.tex
> +++ b/device-types/net/device-conformance.tex
> @@ -15,4 +15,5 @@
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Receive-side scaling (RSS) / RSS processing}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing}
>  \item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Inner Header Hash}
> +\item \ref{devicenormative:Device Types / Network Device / Device Operation / Control Virtqueue / Member Device}
>  \end{itemize}
> -- 
> 2.32.0.3.g01195cf9f