[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] RE: [PATCH v6] virtio-net: support the virtqueue coalescing moderation
Hi, Parav. å 2023/2/22 äå12:13, Parav Pandit åé:
From: Heng Qi <hengqi@linux.alibaba.com> Sent: Tuesday, February 21, 2023 10:22 PM +\item[VIRTIO_NET_F_VQ_NOTF_COAL(52)] Device supports virtqueue notification coalescing. +s/notification/notifications should be plural as multiple notifications are coalesced like the below description of VIRTIO_NET_F_NOTF_COAL.
I see, but I saw the usage of "notification coalesing" on the professional technical articles exist as well.
I can modify it if you want to be consistent with the following.:)
\item[VIRTIO_NET_F_NOTF_COAL(53)] Device supports notifications coalescing. \item[VIRTIO_NET_F_GUEST_USO4 (54)] Driver can receive USOv4 packets. @@ -139,6 +141,7 @@ \subsubsection{Feature bit requirements}\label{sec:Device Types / Network Device \item[VIRTIO_NET_F_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \item[VIRTIO_NET_F_RSC_EXT] Requires VIRTIO_NET_F_HOST_TSO4 or VIRTIO_NET_F_HOST_TSO6. \item[VIRTIO_NET_F_RSS] Requires VIRTIO_NET_F_CTRL_VQ. +\item[VIRTIO_NET_F_VQ_NOTF_COAL] Requires VIRTIO_NET_F_CTRL_VQ. \end{description} \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits} @@ -1508,6 +1511,14 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi If the VIRTIO_NET_F_NOTF_COAL feature is negotiated, the driver can send control commands for dynamically changing the coalescing parameters. +If the VIRTIO_NET_F_VQ_NOTF_COAL feature is negotiated: +\begin{itemize} +\item a driver can send a VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command to set coalescing parameters of a given + enabled transmit/receive virtqueue. +\item a driver can send a VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET command to a device, and the device responds with + coalescing parameters of a given enabled transmit/receive virtqueue. +\end{itemize} + \begin{note} The behavior of the device in response to these commands is best-effort: the device may generate notifications more or less frequently than specified. @@ -1519,25 +1530,76 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi le32 max_usecs; }; +struct virtio_net_ctrl_coal_vq { + le16 vqn; + le16 reserved; + struct virtio_net_ctrl_coal coal; +}; + #define VIRTIO_NET_CTRL_NOTF_COAL 6 #define VIRTIO_NET_CTRL_NOTF_COAL_TX_SET 0 #define VIRTIO_NET_CTRL_NOTF_COAL_RX_SET 1 + #define VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET 2 #define + VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET 3 \end{lstlisting} +The VIRTIO_NET_CTRL_NOTF_COAL_TX_SET and +VIRTIO_NET_CTRL_NOTF_COAL_RX_SET commands use the virtio_net_ctrl_coal +structure to set \field{max_usecs} and \field{max_packets} for all transmit/receive virtqueues. + +The VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command uses the +virtio_net_ctrl_coal_vq structure to set \field{max_usecs} and \field{max_packets} for the supplied virtqueue number \field{vqn}. + +The VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET command gets the values of +\field{max_usecs} and \field{max_packets} of the specified virtqueue +from the device by setting \field{vqn} in the virtio_net_ctrl_coal_vq structure. + +# Read/Write attributes for coalescing parameters \begin{itemize} \item +For commands VIRTIO_NET_CTRL_NOTF_COAL_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_RX_SET, \field{max_usecs} + and \field{max_packets} are write-only for a driver. +\item For the command VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET, \field{vqn},Virtio spec is using vq number and vq index terminology interchangeably. For example, a new patch about admin vq, uses aq_start_index. MMIO device has vq_index register too. I am inclined to vq_index given current state of spec and new additions by Michael.
Yes, both vq number and vq index exist in the existing spec.
For example this paragraph:
"
\begin{description}
\item [vqn] VQ number to be notified.
\item [next_off] Offset
...
\end{description}
"
Michael, Can you please suggest vqn or vq_index to use?
Let's hear Michael's opinion.
\field{reserved}, \field{max_usecs} + and \field{max_packets} are write-only for a driver. +\item For the command VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET, \field{vqn} and \field{reserved} are write-only+ for a driver, and, \field{max_usecs} and \field{max_packets} are read-only for a driver.Remove this trailing "for a driver", it is a duplicate.
Ok.
+\end{itemize} + Coalescing parameters: \begin{itemize} +\item \field{vqn}: The virtqueue number of an enabled transmit or receive virtqueue. \item \field{max_usecs} for RX: Maximum number of microseconds to delay a RX notification. \item \field{max_usecs} for TX: Maximum number of microseconds to delay a TX notification. \item \field{max_packets} for RX: Maximum number of packets to receive before a RX notification. \item \field{max_packets} for TX: Maximum number of packets to send before a TX notification. \end{itemize} -The class VIRTIO_NET_CTRL_NOTF_COAL has 2 commands: +\field{vqn} points to an enabled transmit/receive virtqueue, and its value satisfies $ 0 \leq vqn < max_virtqueue_pairs \ast 2 $. +This calculation description is not needed here. It is covered somewhere else.
Ok, I'll drop this.
+\field{reserved} is reserved and it is ignored by a device. + +The class VIRTIO_NET_CTRL_NOTF_COAL has 4 commands: \begin{enumerate} -\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: set the \field{max_usecs} and \field{max_packets} parameters for all transmit virtqueues. -\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: set the \field{max_usecs} and \field{max_packets} parameters for all receive virtqueues. +\item VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET: set the \field{max_usecs} and \field{max_packets} parameters for an enabled transmit/receive + virtqueue whose number is \field{vqn}. +\item VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET: the device returns the \field{max_usecs} and \field{max_packets} parameters for an enabled + transmit/receive virtqueue whose number is \field{vqn}. +\item VIRTIO_NET_CTRL_NOTF_COAL_TX_SET: have the same effect as the VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command repeated for each virtqueue of transmitq1\ldots transmitqN. +\item VIRTIO_NET_CTRL_NOTF_COAL_RX_SET: have the same effect as the VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command repeated for each virtqueue of receiveq1\ldots receiveqN. \end{enumerate} +If coalescing parameters are being set, the device applies the last +coalescing parameters received for a virtqueue, regardless of the command used to set the parameters. For example with 2 pairs of virtqueues: +# Command sequence +Each of the following commands sets \field{max_usecs} and \field{max_packets} parameters for virtqueues. +\begin{itemize} +\item Command1: VIRTIO_NET_CTRL_NOTF_COAL_RX_SET sets coalescing parameters for virtqueue0 and virtqueue2, and, virtqueue1 and virtqueue3 retain their previous parameter values. +\item Command2: VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET with \field{vqn} = 0 sets coalescing parameters for virtqueue0, and virtqueue2 retains the values from command1. +\item Command3: VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET with \field{vqn} = 0, the device responds with coalescing parameters of virtqueue0 set by command2. +\item Command4: VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET with \field{vqn} = 1 sets coalescing parameters for virtqueue1, and virtqueue3 retains its previous values. +\item Command5: VIRTIO_NET_CTRL_NOTF_COAL_TX_SET sets coalescing parameters for virtqueue1 and virtqueue3, and overrides the values set by command4. +\item Command6: VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET with \field{vqn} = 1, the device responds with coalescing parameters of virtqueue1 set by command5. +\end{itemize} + \subparagraph{Operation}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / Operation} The device sends a used buffer notification once the notification conditions are met and if the notifications are not suppressed as explained in \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / Used Buffer Notification Suppression}. @@ -1549,6 +1611,16 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi When the device has \field{max_usecs} = 0 or \field{max_packets} = 0, the notification conditions are met after every packet received/sent. +When a device receives a command of the VIRTIO_NET_CTRL_NOTF_COAL class +to set a coalescing parameter, it may set the parameter to a value close to a power of 2. For example: +If the device receives \field{max_usecs} = 7 from the VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command, it may set \field{max_usecs} = 8 for a given enabled virtqueue. + +When the device receives the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET and +VIRTIO_NET_CTRL_NOTF_COAL_RX_SET commands, it saves the values of +coalescing parameters as global values, and the VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command does not change the global values. If the device is reset, the global values will be set to 0. + +When a virtqueue is reset, its coalescing parameters are set to the global values. +A VQ reset operation disables (or destroys) the VQ in the device. Hence, a vq under reset doesn't have any valid parameters. Therefore, above wording should be, When a virtqueue is enabled after it is reset, its coalescing parameters are set to global values as configured by VIRTIO_NET_CTRL_NOTF_COAL_TX_SET or VIRTIO_NET_CTRL_NOTF_COAL_RX_SET.
Thanks for your suggestion! I think the following description can clarify?âWhen a virtqueue is enabled after it is reset, its coalescing parameters are set to global values.â
Because the global values have been mentioned above: " +When the device receives the VIRTIO_NET_CTRL_NOTF_COAL_TX_SET and +VIRTIO_NET_CTRL_NOTF_COAL_RX_SET commands, it saves the values of+coalescing parameters as global values, and the VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command does not change the global values.
If the device is reset, the global values will be set to 0. "and it may be 0 after device reset, or it may be the values set by VIRTIO_NET_CTRL_NOTF_COAL_RX_SET or VIRTIO_NET_CTRL_NOTF_COAL_TX_SET.
Thank!
\subparagraph{RX Example}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing / RX Example} If, for example: @@ -1585,11 +1657,29 @@ \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Devi \drivernormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} -If the VIRTIO_NET_F_NOTF_COAL feature has not been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands. +If neither the VIRTIO_NET_F_NOTF_COAL nor the VIRTIO_NET_F_VQ_NOTF_COAL +feature has been negotiated, the driver MUST NOT issue VIRTIO_NET_CTRL_NOTF_COAL commands. + +A driver MUST ignore the values of coalescing parameters received from the VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET command if a device responds with VIRTIO_NET_ERR. \devicenormative{\subparagraph}{Notifications Coalescing}{Device Types / Network Device / Device Operation / Control Virtqueue / Notifications Coalescing} -A device SHOULD respond to the VIRTIO_NET_CTRL_NOTF_COAL commands with VIRTIO_NET_ERR if it was not able to change the parameters. +A device SHOULD respond to VIRTIO_NET_CTRL_NOTF_COAL_TX_SET and VIRTIO_NET_CTRL_NOTF_COAL_RX_SET commands with VIRTIO_NET_ERR if it was not able to change the parameters. + +A device MUST respond to the VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command with VIRTIO_NET_ERR if it was not able to change the parameters. + +A device MUST respond to VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET and VIRTIO_NET_CTRL_NOTF_COAL_VQ_GET commands with VIRTIO_NET_ERR if the given virtqueue is disabled. + +The VIRTIO_NET_CTRL_NOTF_COAL_TX_SET and +VIRTIO_NET_CTRL_NOTF_COAL_RX_SET commands set coalescing parameters for all transmit/receive virtqueues respectively and values of coalescing parameters are recorded as global values by a device. +The device MUST set the global values of coalescing parameters to 0 after being reset. +The VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET command sets the coalescing parameters for a given enabled virtqueue without changing the global values. + +After disabling and re-enabling a virtqueue, the device MUST revert coalescing parameters of the virtqueue to the global values. + +A device MAY set the coalescing parameter to a value close to a power of 2 value. + +A device MUST ignore \field{reserved}. A device SHOULD NOT send used buffer notifications to the driver if the notifications are suppressed, even if the notification conditions are met. -- 2.19.1.6.gb485710b--------------------------------------------------------------------- To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]