Re: [virtio-comment] multibuffer in packed queue

[Krasnov Arseniy <oxffffaa@gmail.com>]

ÑÑ, 13 ÐÐÐ. 2020 Ð. Ð 17:29, Krasnov Arseniy <oxffffaa@gmail.com>:

Ok, i understand about how virtio-net mergeÂworks. I asked about the following thing, consider we have two buffers(two descriptors each):

buffer 1(two descriptors):

| buf_1_1, NEXT, !USED, 1KB | | buf_1_2, !NEXT, !USED, 1KB|

buffer 2(two descriptors):

Â| buf_2_1, NEXT, !USED, 1KB | | buf_2_2, !NEXT, !USED, 1KB|Â

Both four descriptors placed sequentially in queue: buf_1_1, buf_1_2, buf_2_1, buf_2_2.

Then 3KB network packet arrived and device fills buffers:

 | buf_1_1, NEXT, USED, 0KB | | buf_1_2, !NEXT, USED, 0KB|ÂÂ



Â| buf_2_1, NEXT, USED, 0KB | | buf_2_2, !NEXT, USED, 512B|Â

Is device allowed to flip NEXT bit of buf_1_2, two make chain of four descriptors(linking buf_1_2 and buf_2_1)?

ÑÑ, 13 ÐÐÐ. 2020 Ð. Ð 15:16, Stefan Hajnoczi <stefanha@redhat.com>:

On Thu, Aug 13, 2020 at 10:57:52AM +0300, Krasnov Arseniy wrote:


> Hellow, I've got question about chapter 2.7.9:


>


> *Â 2.7.9 Multi-buffer requests Some devices combine multiple buffers as


> part of processing of a single request. These devices always mark the


> descriptor corresponding to the first buffer in the request used after the


> rest of the descriptors (corresponding to rest of the buffers) in the


> request - which follow the first descriptor in ring order - has been marked


> used and written out into the ring. This guarantees that the driver will


> never observe a partial request in the ring. *


>


> Does it mean, that device can use sequence(chain) of descriptor unil


> descriptor with VIRTQ_DESC_F_NEXTÂ == 0 bit is found, OR device can use


> descriptors as many as it needs, don't care about VIRTQ_DESC_F_NEXT bit,


> and finally set VIRTQ_DESC_F_NEXT bit in all used descriptors.





2.7.9 talks about 3 concepts:


1. Requests (highest-level concept)


2. Buffers


3. Descriptors (lowest-level concept)





They are distinct. A request is something like a received packet from a


network card. A buffer is a descriptor chain (1 or more descriptors with


the F_NEXT bit set). A descriptor is a single memory [start, end) range


that the device may read from or write to.





The paragraph descibes how some devices complete one request by


combining multiple buffers. If a 8KB network packet arrives on a NIC and


the driver provided two 4KB buffers consisting of 1 4KB descriptor each


then the device may "merge" those two buffers (see virtio-net for


details). When doing so it must ensure that the first 4KB buffer of the


packet is marked used and written out to the ring *after* the second 4KB


buffer so that the driver sees the beginning of the network packet and


not the second half with the first half missing.





Merging buffers is done at the device type-level (e.g. virtio-net), not


at a virtqueue-level (i.e. core virtio code). The device type must


specify exactly how merging works and device type-specific header fields are


be used by the device to indicate which buffers have been merged into a


single request (see virtio-net's VIRTIO_NET_F_MRG_RXBUF feature bit and


num_buffers header field).





> Also, in packed queue device can't modify descriptors except USED bit.





No, the device *must* overwrite descriptors in the packed ring. Imagine


a device that completes buffers out-of-order:





1. The driver submits two buffers (F_NEXT is not set because a chain of


 Âdescriptors would make the ASCII diagram more complex) and waits for


 Âthe device:





 | buf#1 !USED LEN=4KB | buf#2 !USED LEN=4KB |





2. The device completes buf#2 first (1KB of the 4KB descriptor was


 Âwritten) and writes the used descriptor to the ring:





 | buf#2 USED LEN=1KB | buf#2 !USED LEN=4KB |





3. Let's say the driver waits for buf#1 to complete (3KB of the 4KB


 Âdescriptor was written):





 | buf#2 USED LEN=1KB | buf#1 USED LEN=3KB |





You can see that the ring was overwritten by the device and the


descriptors were modified (e.g. the 4KB avail length was replaced by the


1KB used length value).





> In


> split queue, device also can't modify descriptor(it touches used buffer


> with index and length fields). Is that true?





Yes, the device does not modify split queue descriptors. It only fills


in used ring elements and updates the used ring index.





Stefan