Re: [virtio-dev] Re: [RFC PATCH v6] virtio-video: Add virtio video device specification

[Alexander Gordeev <alexander.gordeev@opensynergy.com>]

On 03.05.23 17:11, Alex BennÃe wrote:
> Cornelia Huck <cohuck@redhat.com> writes:
>
> > On Fri, Apr 28 2023, Alexander Gordeev <alexander.gordeev@opensynergy.com> wrote:
> >
> > > On 27.04.23 15:16, Alexandre Courbot wrote:
> > > > But in any case, that's irrelevant to the guest-host interface, and I
> > > > think a big part of the disagreement stems from the misconception that
> > > > V4L2 absolutely needs to be used on either the guest or the host,
> > > > which is absolutely not the case.
> > >
> > > I understand this, of course. I'm arguing, that it is harder to
> > > implement it, get it straight and then maintain it over years. Also it
> > > brings limitations, that sometimes can be workarounded in the virtio
> > > spec, but this always comes at a cost of decreased readability and
> > > increased complexity. Overall it looks clearly as a downgrade compared
> > > to virtio-video for our use-case. And I believe it would be the same for
> > > every developer, that has to actually implement the spec, not just do
> > > the pass through. So if we think of V4L2 UAPI pass through as a
> > > compatibility device (which I believe it is), then it is fine to have
> > > both and keep improving the virtio-video, including taking the best
> > > ideas from the V4L2 and overall using it as a reference to make writing
> > > the driver simpler.
> >
> > Let me jump in here and ask another question:
> >
> > Imagine that, some years in the future, somebody wants to add a virtio
> > device for handling video encoding/decoding to their hypervisor.
> >
> > Option 1: There are different devices to chose from. How is the person
> > implementing this supposed to pick a device? They might have a narrow
> > use case, where it is clear which of the devices is the one that needs to
> > be supported; but they also might have multiple, diverse use cases, and
> > end up needing to implement all of the devices.
> >
> > Option 2: There is one device with various optional features. The person
> > implementing this can start off with a certain subset of features
> > depending on their expected use cases, and add to it later, if needed;
> > but the upfront complexity might be too high for specialized use cases.
> >
> > Leaving concrete references to V4L2 out of the picture, we're currently
> > trying to decide whether our future will be more like Option 1 or Option
> > 2, with their respective trade-offs.
> >
> > I'm slightly biased towards Option 2; does it look feasible at all, or
> > am I missing something essential here? (I had the impression that some
> > previous confusion had been cleared up; apologies in advance if I'm
> > misrepresenting things.)
> >
> > I'd really love to see some kind of consensus for 1.3, if at all
> > possible :)
>
> I think feature discovery and extensibility is a key part of the VirtIO
> paradigm which is why I find the virtio-v4l approach limiting. By
> pegging the device to a Linux API we effectively limit the growth of the
> device specification to as fast as the Linux API changes. I'm not fully
> immersed in v4l but I don't think it is seeing any additional features
> developed for it and its limitations for camera are one of the reasons
> stuff is being pushed to userspace in solutions like libcamera:
>
>    How is libcamera different from V4L2?
>
>    We see libcamera as a continuation of V4L2. One that can more easily
>    handle the recent advances in hardware design. As embedded cameras have
>    developed, all of the complexity has been pushed on to the developers.
>    With libcamera, all of that complexity is simplified and a single model
>    is presented to application developers.
>
> That said its not totally our experience to have virtio devices act as
> simple pipes for some higher level protocol. The virtio-gpu spec says
> very little about the details of how 3D devices work and simply offers
> an opaque pipe to push a (potentially propriety) command stream to the
> back end. As far as I'm aware the proposals for Vulkan and Wayland
> device support doesn't even offer a feature bit but simply changes the
> graphics stream type in the command packets.

I'd like to note, that virtio-v4l2 is not going to be a simple pipe for
V4L2 UAPI already AFAIU, because it is going to have some patches for
the forwarded protocol in the virtio spec. AFAIK virtio-gpu doesn't do this.

> We could just offer a VIRTIO_VIDEO_F_V4L feature bit, document it as
> incompatible with other feature bits and make that the baseline
> implementation but it's not really in the spirit of what VirtIO is
> trying to achieve.

Thank you for your input!

--
Alexander Gordeev
Senior Software Engineer

OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin

Phone: +49 30 60 98 54 0 - 88
Fax: +49 (30) 60 98 54 0 - 99
EMail: alexander.gordeev@opensynergy.com

www.opensynergy.com

Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
GeschÃftsfÃhrer/Managing Director: RÃgis Adjamah

Please mind our privacy notice<https://www.opensynergy.com/datenschutzerklaerung/privacy-notice-for-business-partners-pursuant-to-article-13-of-the-general-data-protection-regulation-gdpr/> pursuant to Art. 13 GDPR. // Unsere Hinweise zum Datenschutz gem. Art. 13 DSGVO finden Sie hier.<https://www.opensynergy.com/de/datenschutzerklaerung/datenschutzhinweise-fuer-geschaeftspartner-gem-art-13-dsgvo/>