Re: [virtio-dev] Re: [RFC PATCH v6] virtio-video: Add virtio video device specification

[Alexander Gordeev <alexander.gordeev@opensynergy.com>]

On 03.05.23 16:04, Cornelia Huck wrote:
> On Fri, Apr 28 2023, Alexander Gordeev <alexander.gordeev@opensynergy.com> wrote:
>
> > On 27.04.23 15:16, Alexandre Courbot wrote:
> > > But in any case, that's irrelevant to the guest-host interface, and I
> > > think a big part of the disagreement stems from the misconception that
> > > V4L2 absolutely needs to be used on either the guest or the host,
> > > which is absolutely not the case.
> >
> > I understand this, of course. I'm arguing, that it is harder to
> > implement it, get it straight and then maintain it over years. Also it
> > brings limitations, that sometimes can be workarounded in the virtio
> > spec, but this always comes at a cost of decreased readability and
> > increased complexity. Overall it looks clearly as a downgrade compared
> > to virtio-video for our use-case. And I believe it would be the same for
> > every developer, that has to actually implement the spec, not just do
> > the pass through. So if we think of V4L2 UAPI pass through as a
> > compatibility device (which I believe it is), then it is fine to have
> > both and keep improving the virtio-video, including taking the best
> > ideas from the V4L2 and overall using it as a reference to make writing
> > the driver simpler.
>
> Let me jump in here and ask another question:
>
> Imagine that, some years in the future, somebody wants to add a virtio
> device for handling video encoding/decoding to their hypervisor.
>
> Option 1: There are different devices to chose from. How is the person
> implementing this supposed to pick a device? They might have a narrow
> use case, where it is clear which of the devices is the one that needs to
> be supported; but they also might have multiple, diverse use cases, and
> end up needing to implement all of the devices.

I think in this case virtio-v4l2 should be used as a compatibility
device exclusively. This means discouraging increasing its complexity
even more with more patches in the spec. virtio-video should eventually
cover all the use-cases of V4L2, so I think it is reasonable to use it
in both complex use-cases and in simple use-cases, where there is no
decoder/encoder V4L2 device on the host.

> Option 2: There is one device with various optional features. The person
> implementing this can start off with a certain subset of features
> depending on their expected use cases, and add to it later, if needed;
> but the upfront complexity might be too high for specialized use cases.

In this case I'd prefer to have the simpler device first, that is the
current virtio-video, then to add features incrementally using feature
flags and taking into account the virtualization context. V4L2 is a
complex thing from a different context. They already tried to carve out
some of the use-cases like stateful decoder/encoder API, but this work
is not finished (struct v4l2_buffer can serve as an evidence). This is
like dissecting a monolith. Also it has to be patched to make it more
appropriate for virtualization (we can see this in Alexandre's PoC already).

> Leaving concrete references to V4L2 out of the picture, we're currently
> trying to decide whether our future will be more like Option 1 or Option
> 2, with their respective trade-offs.

I'd like to rely on opinions of people, who know more about virtio
development and goals. I would be happy to present or reiterate my
arguments to anyone interested if necessary.

> I'm slightly biased towards Option 2; does it look feasible at all, or
> am I missing something essential here? (I had the impression that some
> previous confusion had been cleared up; apologies in advance if I'm
> misrepresenting things.)

Indeed some of the previous confusion has been cleared up. But not the
key thing. Alexandre still claims, that this patched V4L2 UAPI pass
through is only marginally more complex, for example. I don't agree with
this and I have evidence. We haven't finished discussing this evidence.

> I'd really love to see some kind of consensus for 1.3, if at all
> possible :)

--
Alexander Gordeev
Senior Software Engineer

OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin

Phone: +49 30 60 98 54 0 - 88
Fax: +49 (30) 60 98 54 0 - 99
EMail: alexander.gordeev@opensynergy.com

www.opensynergy.com

Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B
GeschÃftsfÃhrer/Managing Director: RÃgis Adjamah

Please mind our privacy notice<https://www.opensynergy.com/datenschutzerklaerung/privacy-notice-for-business-partners-pursuant-to-article-13-of-the-general-data-protection-regulation-gdpr/> pursuant to Art. 13 GDPR. // Unsere Hinweise zum Datenschutz gem. Art. 13 DSGVO finden Sie hier.<https://www.opensynergy.com/de/datenschutzerklaerung/datenschutzhinweise-fuer-geschaeftspartner-gem-art-13-dsgvo/>