OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

virtio message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: [PATCH v2 1/2] net: add _F_MQ support


VIRTIO-49

Includes git commits:

3c600996f641614d3720c94dd52155aaaba670fa
    virtio-spec: fix two typos
commit 67023431c8796bc430ec0a79b15bab57e2e0f1f6
    virtio-spec: virtio network device multiqueue support
commit a02d91f8729b4a333d525015d22138a86ce9b644
    net: add note that you can defer rx queue init until mq enable.

Reported-by: Francesco Fusco <ffusco@redhat.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
 content.tex                              | 93 ++++++++++++++++++++++++++++----
 virtio-v1.0-wd01-part1-specification.txt | 90 +++++++++++++++++++++++++++----
 2 files changed, 165 insertions(+), 18 deletions(-)

diff --git a/content.tex b/content.tex
index 37c8257..637ca2b 100644
--- a/content.tex
+++ b/content.tex
@@ -2228,9 +2228,12 @@ features.
 
 \subsection{Virtqueues}\label{sec:Device Types / Network Device / Virtqueues}
 
- 0:receiveq. 1:transmitq. 2:controlq
+ 0:receiveq0. 1:transmitq0. .... 2N:receivqN. 2N+1: transmitqN. 2N+2:controlq
 
- Virtqueue 2 only exists if VIRTIO_NET_F_CTRL_VQ set.
+ N=0 if VIRTIO_NET_F_MQ is not negotiated, otherwise N is derived
+ from max_virtqueue_pairs control field.
+
+ controlq only exists if VIRTIO_NET_F_CTRL_VQ set.
 
 \subsection{Feature bits}\label{sec:Device Types / Network Device / Feature bits}
 
@@ -2273,6 +2276,9 @@ features.
   VIRTIO_NET_F_GUEST_ANNOUNCE(21) Guest can send gratuitous
     packets.
 
+  VIRTIO_NET_F_MQ(22) Device supports multiqueue with automatic
+    receive steering.
+
 \subsubsection{Legacy Interface: Feature bits}\label{sec:Device Types / Network Device / Feature bits / Legacy Interface: Feature bits}
 VIRTIO_NET_F_GSO (6) Device handles packets with any GSO type.
 
@@ -2282,7 +2288,7 @@ were required.
 
 \subsection{Device configuration layout}\label{sec:Device Types / Network Device / Device configuration layout}
 
-Two configuration fields are currently defined. The mac address field
+Three configuration fields are currently defined. The mac address field
 always exists (though is only valid if VIRTIO_NET_F_MAC is set), and
 the status field only exists if VIRTIO_NET_F_STATUS is set. Two
 read-only bits are currently defined for the status field:
@@ -2291,10 +2297,20 @@ VIRTIO_NET_S_LINK_UP and VIRTIO_NET_S_ANNOUNCE.
 \begin{lstlisting}
 	#define VIRTIO_NET_S_LINK_UP	1
 	#define VIRTIO_NET_S_ANNOUNCE	2
+\end{lstlisting}
+
+The following read-only field, max_virtqueue_pairs only exists if
+VIRTIO_NET_F_MQ is set. This field specifies the maximum number
+of each of transmit and receive virtqueues (receiveq0..receiveqN
+and transmitq0..transmitqN respectively;
+ N=max_virtqueue_pairs - 1) that can be configured once VIRTIO_NET_F_MQ
+is negotiated.  Legal values for this field are 1 to 0x8000.
 
+\begin{lstlisting}
 	struct virtio_net_config {
 		u8 mac[6];
 		le16 status;
+		u16 max_virtqueue_pairs;
 	};
 \end{lstlisting}
 
@@ -2306,7 +2322,9 @@ native endian of the guest rather than (necessarily) little-endian.
 \subsection{Device Initialization}\label{sec:Device Types / Network Device / Device Initialization}
 
 1. The initialization routine should identify the receive and
-  transmission virtqueues.
+  transmission virtqueues, up to N+1 of each kind. If
+  VIRTIO_NET_F_MQ feature bit is negotiated,
+  N=max_virtqueue_pairs-1, otherwise identify N=0.
 
 2. If the VIRTIO_NET_F_MAC feature bit is set, the configuration
   space “mac” entry indicates the “physical” address of the the
@@ -2321,7 +2339,17 @@ native endian of the guest rather than (necessarily) little-endian.
   status can be read from the bottom bit of the “status” config
   field. Otherwise, the link should be assumed active.
 
-5. The receive virtqueue should be filled with receive buffers.
+5. Only receiveq0, transmitq0 and controlq are used by default.
+  To use more queues driver must negotiate the VIRTIO_NET_F_MQ
+  feature; initialize up to max_virtqueue_pairs of each of
+  transmit and receive queues;
+  execute_VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command specifying the
+  number of the transmit and receive queues that is going to be
+  used and wait until the device consumes the controlq buffer and
+  acks this command.
+  The receive virtqueue should be filled with receive buffers
+  before multiqueue is activated
+  (see \ref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}~\nameref{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}).
   This is described in detail below in “Setting Up Receive
   Buffers”.
 
@@ -2361,9 +2389,10 @@ if both guests are amenable.
 
 \subsection{Device Operation}\label{sec:Device Types / Network Device / Device Operation}
 
-Packets are transmitted by placing them in the transmitq, and
-buffers for incoming packets are placed in the receiveq. In each
-case, the packet itself is preceeded by a header:
+Packets are transmitted by placing them in the
+transmitq0..transmitqN, and buffers for incoming packets are
+placed in the receiveq0..receiveqN. In each case, the packet
+itself is preceeded by a header:
 
 \begin{lstlisting}
 	struct virtio_net_hdr {
@@ -2486,6 +2515,9 @@ elements.
 If VIRTIO_NET_F_MRG_RXBUF is negotiated, each buffer must be at
 least the size of the struct virtio_net_hdr.
 
+If VIRTIO_NET_F_MQ is negotiated, each of receiveq0...receiveqN
+that will be used should be populated with receive buffers.
+
 \paragraph{Packet Receive Interrupt}\label{sec:Device Types / Network Device / Device Operation / Setting Up Receive Buffers / Packet Receive Interrupt}
 
 When a packet is copied into a buffer in the receiveq, the
@@ -2522,7 +2554,7 @@ Processing packet involves:
 
 \subsubsection{Control Virtqueue}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue}
 
-The driver uses the control virtqueue (if VIRTIO_NET_F_VTRL_VQ is
+The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
 negotiated) to send commands to manipulate various features of
 the device which would not easily map into the configuration
 space.
@@ -2646,6 +2678,49 @@ Processing this notification involves:
 2. Sending VIRTIO_NET_CTRL_ANNOUNCE_ACK command through control
   vq.
 
+\paragraph{Automatic receive steering in multiqueue mode}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Automatic receive steering in multiqueue mode}
+
+If the driver negotiates the VIRTIO_NET_F_MQ feature bit (depends
+on VIRTIO_NET_F_CTRL_VQ), it can transmit outgoing packets on one
+of the multiple transmitq0..transmitqN and ask the device to
+queue incoming packets into one the multiple receiveq0..receiveqN
+depending on the packet flow.
+
+\begin{lstlisting}
+	struct virtio_net_ctrl_mq {
+		u16 virtqueue_pairs;
+	};
+
+	#define VIRTIO_NET_CTRL_MQ    4
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET        0
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MIN        1
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX        0x8000
+\end{lstlisting}
+
+Multiqueue is disabled by default. Driver enables multiqueue by
+executing the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command, specifying
+the number of the transmit and receive queues that will be used;
+thus transmitq0..transmitqn and receiveq0..receiveqn where
+n=virtqueue_pairs-1 will be used. All these virtqueues must have
+been pre-configured in advance. The range of legal values for the
+virtqueue_pairs field is between 1 and max_virtqueue_pairs.
+
+When multiqueue is enabled, device uses automatic receive steering
+based on packet flow.Programming of the receive steering
+classificator is implicit. Transmitting a packet of a specific
+flow on transmitqX will cause incoming packets for this flow to
+be steered to receiveqX. For uni-directional protocols, or where
+no packets have been transmitted yet, device will steer a packet
+to a random queue out of the specified receiveq0..receiveqn.
+
+Multiqueue is disabled by setting virtqueue_pairs = 1 (this is
+the default). After the command is consumed by the device, the
+device will not steer new packets on virtqueues
+receveq1..receiveqN (i.e. other than receiveq0) nor read from
+transmitq1..transmitqN (i.e. other than transmitq0); accordingly,
+driver should not transmit new packets on virtqueues other than
+transmitq0.
+
 \paragraph{Offloads State Configuration}\label{sec:Device Types / Network Device / Device Operation / Control Virtqueue / Offloads State Configuration}
 
 If the VIRTIO_NET_F_CTRL_GUEST_OFFLOADS feature is negotiated, the driver can
diff --git a/virtio-v1.0-wd01-part1-specification.txt b/virtio-v1.0-wd01-part1-specification.txt
index ba3ecae..bfbece3 100644
--- a/virtio-v1.0-wd01-part1-specification.txt
+++ b/virtio-v1.0-wd01-part1-specification.txt
@@ -2309,9 +2309,12 @@ features.
 2.4.1.2. Virtqueues
 ------------------
 
- 0:receiveq. 1:transmitq. 2:controlq
+ 0:receiveq0. 1:transmitq0. .... 2N:receivqN. 2N+1: transmitqN. 2N+2:controlq
 
- Virtqueue 2 only exists if VIRTIO_NET_F_CTRL_VQ set.
+ N=0 if VIRTIO_NET_F_MQ is not negotiated, otherwise N is derived
+ from max_virtqueue_pairs control field.
+
+ controlq only exists if VIRTIO_NET_F_CTRL_VQ set.
 
 2.4.1.3. Feature bits
 --------------------
@@ -2355,6 +2358,9 @@ features.
   VIRTIO_NET_F_GUEST_ANNOUNCE(21) Guest can send gratuitous
     packets.
 
+  VIRTIO_NET_F_MQ(22) Device supports multiqueue with automatic
+    receive steering.
+
 2.4.1.3.1. Legacy Interface: Feature bits
 --------------------
 VIRTIO_NET_F_GSO (6) Device handles packets with any GSO type.
@@ -2366,7 +2372,7 @@ were required.
 100.4.1.4. Device configuration layout
 ---------------------
 
-Two configuration fields are currently defined. The mac address field
+Three configuration fields are currently defined. The mac address field
 always exists (though is only valid if VIRTIO_NET_F_MAC is set), and
 the status field only exists if VIRTIO_NET_F_STATUS is set. Two
 read-only bits are currently defined for the status field:
@@ -2375,9 +2381,17 @@ VIRTIO_NET_S_LINK_UP and VIRTIO_NET_S_ANNOUNCE.
 	#define VIRTIO_NET_S_LINK_UP	1
 	#define VIRTIO_NET_S_ANNOUNCE	2
 
+The following read-only field, max_virtqueue_pairs only exists if
+VIRTIO_NET_F_MQ is set. This field specifies the maximum number
+of each of transmit and receive virtqueues (receiveq0..receiveqN
+and transmitq0..transmitqN respectively;
+ N=max_virtqueue_pairs - 1) that can be configured once VIRTIO_NET_F_MQ
+is negotiated.  Legal values for this field are 1 to 0x8000.
+
 	struct virtio_net_config {
 		u8 mac[6];
 		le16 status;
+		u16 max_virtqueue_pairs;
 	};
 
 100.4.1.4.1. Legacy Interface: Device configuration layout
@@ -2390,7 +2404,9 @@ native endian of the guest rather than (necessarily) little-endian.
 -----------------------------
 
 1. The initialization routine should identify the receive and
-  transmission virtqueues.
+  transmission virtqueues, up to N+1 of each kind. If
+  VIRTIO_NET_F_MQ feature bit is negotiated,
+  N=max_virtqueue_pairs-1, otherwise identify N=0.
 
 2. If the VIRTIO_NET_F_MAC feature bit is set, the configuration
   space “mac” entry indicates the “physical” address of the the
@@ -2405,7 +2421,17 @@ native endian of the guest rather than (necessarily) little-endian.
   status can be read from the bottom bit of the “status” config
   field. Otherwise, the link should be assumed active.
 
-5. The receive virtqueue should be filled with receive buffers.
+5. Only receiveq0, transmitq0 and controlq are used by default.
+  To use more queues driver must negotiate the VIRTIO_NET_F_MQ
+  feature; initialize up to max_virtqueue_pairs of each of
+  transmit and receive queues;
+  execute_VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command specifying the
+  number of the transmit and receive queues that is going to be
+  used and wait until the device consumes the controlq buffer and
+  acks this command.
+  The receive virtqueue should be filled with receive buffers
+  before multiqueue is activated
+  (see "2.4.1.5.3.100. Automatic receive steering in multiqueue mode").
   This is described in detail below in “Setting Up Receive
   Buffers”.
 
@@ -2433,9 +2459,10 @@ native endian of the guest rather than (necessarily) little-endian.
 2.4.1.5. Device Operation
 ------------------------
 
-Packets are transmitted by placing them in the transmitq, and
-buffers for incoming packets are placed in the receiveq. In each
-case, the packet itself is preceeded by a header:
+Packets are transmitted by placing them in the
+transmitq0..transmitqN, and buffers for incoming packets are
+placed in the receiveq0..receiveqN. In each case, the packet
+itself is preceeded by a header:
 
 	struct virtio_net_hdr {
 	#define VIRTIO_NET_HDR_F_NEEDS_CSUM    1
@@ -2538,6 +2565,9 @@ buffer in the receive queue needs to be at least this length [20]
 If VIRTIO_NET_F_MRG_RXBUF is negotiated, each buffer must be at
 least the size of the struct virtio_net_hdr.
 
+If VIRTIO_NET_F_MQ is negotiated, each of receiveq0...receiveqN
+that will be used should be populated with receive buffers.
+
 2.4.1.5.2.1. Packet Receive Interrupt
 ------------------------------------
 
@@ -2576,7 +2606,7 @@ Processing packet involves:
 2.4.1.5.3. Control Virtqueue
 ---------------------------
 
-The driver uses the control virtqueue (if VIRTIO_NET_F_VTRL_VQ is
+The driver uses the control virtqueue (if VIRTIO_NET_F_CTRL_VQ is
 negotiated) to send commands to manipulate various features of
 the device which would not easily map into the configuration
 space.
@@ -2692,6 +2722,48 @@ Processing this notification involves:
 2. Sending VIRTIO_NET_CTRL_ANNOUNCE_ACK command through control
   vq.
 
+2.4.1.5.3.100. Automatic receive steering in multiqueue mode
+--------------------------
+
+If the driver negotiates the VIRTIO_NET_F_MQ feature bit (depends
+on VIRTIO_NET_F_CTRL_VQ), it can transmit outgoing packets on one
+of the multiple transmitq0..transmitqN and ask the device to
+queue incoming packets into one the multiple receiveq0..receiveqN
+depending on the packet flow.
+
+	struct virtio_net_ctrl_mq {
+		u16 virtqueue_pairs;
+	};
+
+	#define VIRTIO_NET_CTRL_MQ    4
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET        0
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MIN        1
+	 #define VIRTIO_NET_CTRL_MQ_VQ_PAIRS_MAX        0x8000
+
+Multiqueue is disabled by default. Driver enables multiqueue by
+executing the VIRTIO_NET_CTRL_MQ_VQ_PAIRS_SET command, specifying
+the number of the transmit and receive queues that will be used;
+thus transmitq0..transmitqn and receiveq0..receiveqn where
+n=virtqueue_pairs-1 will be used. All these virtqueues must have
+been pre-configured in advance. The range of legal values for the
+virtqueue_pairs field is between 1 and max_virtqueue_pairs.
+
+When multiqueue is enabled, device uses automatic receive steering
+based on packet flow.Programming of the receive steering
+classificator is implicit. Transmitting a packet of a specific
+flow on transmitqX will cause incoming packets for this flow to
+be steered to receiveqX. For uni-directional protocols, or where
+no packets have been transmitted yet, device will steer a packet
+to a random queue out of the specified receiveq0..receiveqn.
+
+Multiqueue is disabled by setting virtqueue_pairs = 1 (this is
+the default). After the command is consumed by the device, the
+device will not steer new packets on virtqueues
+receveq1..receiveqN (i.e. other than receiveq0) nor read from
+transmitq1..transmitqN (i.e. other than transmitq0); accordingly,
+driver should not transmit new packets on virtqueues other than
+transmitq0.
+
 2.4.1.5.3.5. Offloads State Configuration
 -------------------------------------
 
-- 
MST



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]