Re: [virtio-comment] [PATCH v5] virtio-net: Add support for the flexible driver notification structure

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Mar 11, 2020 at 06:43:41PM +0100, Halil Pasic wrote:
> On Tue, 10 Mar 2020 08:42:14 +0000
> Vitaly Mireyno <vmireyno@marvell.com> wrote:
> 
> > Currently, the driver notification (available buffer notification) has a fixed structure.
> > If VIRTIO_F_NOTIFICATION_DATA has been negotiated, it includes: vqn, next_off and next_wrap.
> > If notify_off_multiplier > 0, the VQ number can be derived by the device from the Queue Notify address, so vqn may be redundant.
> 
> May or may not be redundant depending on what transport is used! For the
> Channel I/O transport it is not redundant, because there is no such
> thing like Qeueue Notify address or notify_off_multiplier.

Right note this is all in the PCI chapter.


> We do have space left in GPR3 which host the notification data, but for
> the VIRTIO_NET_F_NOTIF_EXTRA_DATA feature to work with ccw, we would
> need to introduce yet another notification data format.
> 
> For that silicone whose features Vitaly is trying to bring into the
> spec, ccw is very unlikely to matter. But then there was a discussion on
> other future uses, which make me worry.
> 
> BTW the normative statements for VIRTIO_NET_F_NOTIF_EXTRA_DATA seem to
> be all PCI specific. How are other transports supposed to react? Not
> accept the feature?
> 
> Conny, what is your opinion on this new feature and mechanism? 
> 
> Actually my knowledge of PCI is not sufficient to understand why would a
> device play this strange cookie game (making the driver send some device
> specific constant for each notification). But that does not matter.
> 
> I guess there will be kernel patches for this. Or are there some
> already? Can the implementer please put me in cc?
> 
> Regards,
> Halil
> 
> > 
> > Some devices benefit from receiving an additional data with driver notifications. This data can optionally replace the vqn field in the driver notification structure.
> > In its simplest form, it would be sufficient for this data to be a per-device constant value.
> > 
> > Changes from v4 - articles fix-up.
> > 
> > Signed-off-by: Vitaly Mireyno <vmireyno@marvell.com>
> > ---
> >  content.tex | 33 +++++++++++++++++++++++++++++++++
> >  1 file changed, 33 insertions(+)
> > 
> > diff --git a/content.tex b/content.tex
> > index b91a132..08e6111 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -965,6 +965,8 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> >  struct virtio_pci_notify_cap {
> >          struct virtio_pci_cap cap;
> >          le32 notify_off_multiplier; /* Multiplier for queue_notify_off. */
> > +        le16 notify_data; /* Data to be placed in the vqn field */
> > +        le16 padding; /* Pad to a dword */
> >  };
> >  \end{lstlisting}
> >  
> > @@ -984,6 +986,21 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> >  the same Queue Notify address for all queues.
> >  \end{note}
> >  
> > +\field{notify_data} is the data that the driver will set in the \field{vqn}
> > +field in the available buffer notification, if
> > +VIRTIO_NET_F_NOTIF_EXTRA_DATA has been negotiated.
> > +
> > +\begin{note}
> > +If \field{notify_off_multiplier} > 0, the virtqueue number can potentially be
> > +derived by the device from the Queue Notify address, so \field{vqn} may be
> > +redundant. Some devices benefit from receiving additional data with driver
> > +notifications. An example could be a hardware device implementing multiple
> > +protocols (with virtio being one of them), so the extra notification data could
> > +serve as a notification type indication or a protocol indication.
> > +Another example could be using shared hardware memory space for driver
> > +notifications for multiple virtio devices in a trusted environment.
> > +\end{note}
> > +
> >  \devicenormative{\paragraph}{Notification capability}{Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
> >  The device MUST present at least one notification capability.
> >  
> > @@ -1020,6 +1037,9 @@ \subsubsection{Notification structure layout}\label{sec:Virtio Transport Options
> >  cap.length >= queue_notify_off * notify_off_multiplier + 4
> >  \end{lstlisting}
> >  
> > +If the device offers VIRTIO_NET_F_NOTIF_EXTRA_DATA, it MUST set
> > +\field{notify_off_multiplier} > 0 in at least one capability.
> > +
> >  \subsubsection{ISR status capability}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / ISR status capability}
> >  
> >  The VIRTIO_PCI_CAP_ISR_CFG capability
> > @@ -1519,6 +1539,14 @@ \subsubsection{Available Buffer Notifications}\label{sec:Virtio Transport Option
> >  See \ref{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI Device Layout / Notification capability}
> >  for how to calculate the Queue Notify address.
> >  
> > +\drivernormative{\paragraph}{Available Buffer Notifications}{Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Available Buffer Notifications}
> > +The driver SHOULD accept the VIRTIO_NET_F_NOTIF_EXTRA_DATA feature if it has
> > +been offered.
> > +
> > +If VIRTIO_NET_F_NOTIF_EXTRA_DATA has been negotiated and
> > +\field{notify_off_multiplier} > 0, the driver MUST set the \field{vqn} field of
> > +the available buffer notification structure to the \field{notify_data} value.
> > +
> >  \subsubsection{Used Buffer Notifications}\label{sec:Virtio Transport Options / Virtio Over PCI Bus / PCI-specific Initialization And Device Operation / Used Buffer Notifications}
> >  
> >  If a used buffer notification is necessary for a virtqueue, the device would typically act as follows:
> > @@ -2895,6 +2923,10 @@ \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits
> >  \item[VIRTIO_NET_F_CTRL_MAC_ADDR(23)] Set MAC address through control
> >      channel.
> >  
> > +\item[VIRTIO_NET_F_NOTIF_EXTRA_DATA(57)] Driver provides extra data with
> > +    available buffer notifications, to aid in notification processing by the
> > +    device.
> > +
> >  \item[VIRTIO_NET_F_GUEST_HDRLEN(59)] Driver can provide the exact \field{hdr_len}
> >      value. Device benefits from knowing the exact header length.
> >  
> > @@ -2934,6 +2966,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device
> >  \item[VIRTIO_NET_F_CTRL_MAC_ADDR] Requires VIRTIO_NET_F_CTRL_VQ.
> >  \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6.
> >  \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ.
> > +\item[VIRTIO_NET_F_NOTIF_EXTRA_DATA] Requires VIRTIO_F_NOTIFICATION_DATA.
> >  \end{description}
> >  
> >  \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
> > --
> > 
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> > 
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> > 
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/
> >