RE: [virtio-comment] [PATCH requirements 2/7] net-features: Add low latency transmit queue requirements

[Parav Pandit <parav@nvidia.com>]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Tuesday, June 6, 2023 6:26 PM

> > +## 3.2 Low PCI latency virtqueues
> > +### 3.2.1 Low PCI latency tx virtqueue 1. Packet transmit descriptor
> > +should contain data descriptors count without any
> > +   indirection and without any O(N) search to find the end of a packet stream.
> > +   For example, a packet transmit descriptor (called vnet_tx_hdr_desc
> > +   subsequently) to contain a field num_next_desc for the packet stream
> > +   indicating that a packet is located N data descriptors.
> > +
> > +2. Packet transmit descriptor should contain segmentation offload-related
> fields
> > +   without any indirection. For example, packet transmit descriptor to contain
> > +   gso_type, gso_size/mss, header length, csum placement byte offset, and
> > +   csum start.
> > +
> > +3. Packet transmit descriptor should be able to place a small size packet that
> > +   does not have any L4 data after the vnet_tx_hdr_desc in the virtqueue
> memory.
> > +   For example a TCP ack only packet can fit in a descriptor memory which
> > +   otherwise consume more than 25% of metadata to describe the packet.
> 
> For IPv6? With all the crazy options? Are you sure?
> 
IPV6 may not use all option, for example #3 is not useful for IPV6.

> > +
> > +4. Packet transmit descriptor should be able to place a full GSO header (L2 to
> > +   L4) after header descriptor and before data descriptors. For example, the
> > +   GSO header is placed after struct vnet_tx_hdr_desc in the virtqueue
> memory.
> > +   When such a GSO header is positioned adjacent to the packet transmit
> > +   descriptor, and when the GSO header is not aligned to 16B, the following
> > +   data descriptor to start on the 8B aligned boundary.
> > +
> 
> These alignments are weirdly specific.
>
Not sure what it means, 
these alignments are mostly specific on the host side to have aligned writes.
 
> 
> Are these achievable with existing VQ designs? If not this specific part is not
> going to be a network specific thing. which is ok but maybe mention this.
> 
It will be probably a VQ with a type/attributes.

> > +5. An example of the above requirements at high level is:
> > +
> > +```
> > +struct vitio_packed_q_desc {
> > +   /* current desc for reference */
> > +   u64 address;
> > +   u32 len;
> > +   u16 id;
> > +   u16 flags;
> > +};
> > +
> > +/* Constant size header descriptor for tx packets */ struct
> > +vnet_tx_hdr_desc {
> > +   u16 flags; /* indicate how to parse next fields */
> > +   u16 id; /* desc id to come back in completion */
> > +   u8 num_next_desc; /* indicates the number of the next 16B data desc for
> this
> > +		      * buffer.
> > +		      */
> > +   u8 gso_type;
> > +   le16 gso_hdr_len;
> > +   le16 gso_size;
> > +   le16 csum_start;
> > +   le16 csum_offset;
> > +   u8 inline_pkt_len; /* indicates the length of the inline packet after this
> > +		       * desc
> > +		       */
> > +   u8 reserved;
> > +   u8 padding[];
> > +};
> > +
> > +/* Example of a short packet or GSO header placed in the desc section
> > +of the vq  */ struct vnet_tx_small_pkt_desc {
> > +   u8 raw_pkt[128];
> > +};
> > +
> > +/* Example of header followed by data descriptor */ struct
> > +vnet_tx_hdr_desc hdr_desc; struct vnet_data_desc desc[2];
> > +
> > +```
> 
> Seems to assume huge descriptors, apparently of variable size.
> Are fixed size descriptors valuable?
>
Its variable size but size is known upfront. So in a way fixed size.
 
> 
> 
> > +6. Ability to zero pad the transmit completion when the transmit
> > +completion is  shorter than the CPU cache line size.
> > +
> > +7. Ability to place all transmit completion together with it per packet stream
> > +   transmit timestamp using single PCIe transcation.
> 
> I can't decipher the last two ones. Part of it is using non virtio terminology, part
> referring to PCI/CPU unnecessarily, part weird grammar.
> 
Yes, I will rewrite this part. It got messed up.