Re: [virtio-dev] [RFC PATCH v6] virtio-video: Add virtio video device specification

[Cornelia Huck <cohuck@redhat.com>]

On Thu, Jan 12 2023, Alexandre Courbot <acourbot@chromium.org> wrote:

> Hi Cornelia,
>
> On Wed, Jan 11, 2023 at 5:45 PM Cornelia Huck <cohuck@redhat.com> wrote:
>> >> > +
>> >> > +The device MUST set the \field{caps_length} field to a value equal to
>> >> > +the response size of VIRTIO\_VIDEO\_CMD\_DEVICE\_QUERY\_CAPS.
>> >>
>> >> Could the device also support a minimum response size that only supports
>> >> a subset of the caps to be returned? Otherwise, I think caps_length is
>> >> the maximum (or fixed?) length of the query caps response?
>> >
>> > I think this can be replaced by a fixed-size call for getting only one
>> > format at a time. The guest would have to make several of these in
>> > order to obtain the whole set of supported formats, but it would be
>> > easier to parse compared to the large result returned by QUERY_CAP and
>> > simpler overall.
>>
>> How would you implement this? Would the driver do the call repeatedly
>> until no more formats remain (requires the device to track state, and
>> needs a specification on what happens if the driver continues doing the
>> call?) Or would the driver pass in an index, and the device only needs
>> to check for out-of-range?
>
> One peculiarity of codecs is that the pixel formats available may
> depend on the coded format (and sometimes even its resolution). For
> instance, NV12 may be available if you do VP8 at 1080p but not 4K
> H.264.
>
> This means that a single call returning all the coded and pixel
> formats won't be able to convey all the possible subtleties. In
> previous versions of this spec we used bitmasks to associate pixel
> formats to their supported coded formats, but it's a bit of a pain to
> manage while still not bringing the necessary precision.
>
> So here again the safe way is to follow the V4L2 lead: get a list of
> supported bitstream formats (one at a time, using an index with a
> range defined by the configuration space), apply the one we are
> interested in, and query the supported pixel formats for that specific
> bitstream format using the same mechanism. That's more back-and-forth
> between the guest and the host, but it happens before streaming starts
> and better reflects the actual decoding workflow where the client is
> typically only interested in the supported formats for the codec they
> want to decode.

Thanks for the explanation; I agree that this is probably a good way to
handle a complex situation.