[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: VIRTIO_RING_F_INDIRECT_SIZE status
On Mon, Mar 06, 2023 at 12:41:25PM -0500, Michael S. Tsirkin wrote: > On Mon, Mar 06, 2023 at 04:00:37PM +0100, Christian Schoenebeck wrote: > > On Wednesday, March 1, 2023 3:55:57 PM CET Michael S. Tsirkin wrote: > > > On Wed, Mar 01, 2023 at 01:55:14PM +0100, Christian Schoenebeck wrote: > > > > 2.8 Packed Virtqueues > > > > ... > > > > 2.8.5 Scatter-Gather Support [1] > > > > ... > > > > While unusual (most implementations either create all lists solely using > > > > non-indirect descriptors, or always use a single indirect element), if both > > > > features have been negotiated, mixing indirect and non-indirect descriptors > > > > in a ring is valid, as long as each list only contains descriptors of a > > > > given type. > > > > > > > > [1] https://docs.oasis-open.org/virtio/virtio/v1.2/cs01/virtio-v1.2-cs01.html#x1-770005 > > > > > > > > To avoid misapprehensions: the way I understand it, same restrictions apply to > > > > packed queues as split queues, in the sense that you may neither chain several > > > > tables in a single message, nor multi-level nest tables, nor mix a list of > > > > direct descriptors and indirect descriptors on the same level within one > > > > message. So the explicit exception described here, only means you may use > > > > *one* indirect table in one message, while using chained direct descriptors in > > > > another message. But that's it, right? > > > > > > > > > That's my understanding. > > > > > > > > 2. Given this is a lot of work I am trying to find a way to > > > > > make the impact bigger. In particular to cover the use-case > > > > > of limiting s/g to 1k while making queues deeper (with > > > > > or without indirect). For this I proposed: > > > > > > > > > > So I think that given this, we can limit the total number > > > > > of non-indirect descriptors, including non-indirect ones > > > > > in a chain + all the ones in indirect pointer table if any, > > > > > and excluding the indirect descriptor itself, and this > > > > > will address the issue you are describing here, right? > > > > > > > > > > people seemed to be ok with this idea? > > > > > > > > IIUIC it would not make a difference from design perspective from what I > > > > proposed, as virtio currently neither allows to mix, chain or mult-level nest > > > > indirect descriptor tables within a single message), and hence it would just > > > > boil down to adjusting the wording. So yes, it would therefore cover my > > > > intended use case. > > > > > > > > Best regards, > > > > Christian Schoenebeck > > > > > > > > > Sounds good to me. One interesting case is scsi and blk which have > > > a seg_max field. This is defined as > > > > > > \item[\field{seg_max}] is the maximum number of segments that can be in a > > > command. A bidirectional command can include \field{seg_max} input > > > segments and \field{seg_max} output segments. > > > > > > it is never explained what *are* the segments, or how does it > > > interact with VQ depth. Current drivers interpret this > > > strictly and assume that this limits the s/g length but does not > > > allow you to exceed vq size. > > > > > > Do we thus want two limits (for read and write descriptors)? > > > > No opinion on that, as my intended use case was just extending the buffer size > > beyond queue size, not limiting it below queue size. Either way is fine with > > me. > > > > Anyhow, as this now gets broader scope, that also means the suggested flag > > VIRTIO_RING_F_INDIRECT_SIZE needs to be renamed. VIRTIO_RING_F_BUFFER_SIZE? > > > > Best regards, > > Christian Schoenebeck > > > Hmm that's unclear in that it might be in bytes too. > Given blk and scsi call these "segments" how about > VIRTIO_RING_F_SEG_MAX? The VIRTIO equivalent of a "segment" is an "element". I don't think the term "segment" is needed at the VIRTIO device model level since there is already a word for it. I'm confused because VIRTIO_RING_F_BUFFER_SIZE and VIRTIO_RING_F_SEG_MAX mean different things to me and have different units (bytes vs number of segments). I wouldn't worry about virtio-blk/scsi seg_max. Although the segments map to virtqueue elements, seg_max has a specific SCSI/block level meaning related to data transfer and is not about constraints that apply to all virtqueue requests. I/O requests have headers/footers, so they can actually consume more elements than seg_max. Also, there could be non-data transfer requests that happen to consume more than seg_max and the storage controller would be happy with that (e.g. because VIRTIO mandates flexible framing so you could break a request into 1-byte elements). It's confusing the talk about seg_max at the VIRTIO device model level - it's not about virtqueues at all. Stefan
Attachment:
signature.asc
Description: PGP signature
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]