[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] Re: [RFC PATCH v6] virtio-video: Add virtio video device specification
On 03.05.23 17:53, Cornelia Huck wrote:
On Wed, May 03 2023, Alex BennÃe <alex.bennee@linaro.org> wrote:Cornelia Huck <cohuck@redhat.com> writes:On Fri, Apr 28 2023, Alexander Gordeev <alexander.gordeev@opensynergy.com> wrote:On 27.04.23 15:16, Alexandre Courbot wrote:But in any case, that's irrelevant to the guest-host interface, and I think a big part of the disagreement stems from the misconception that V4L2 absolutely needs to be used on either the guest or the host, which is absolutely not the case.I understand this, of course. I'm arguing, that it is harder to implement it, get it straight and then maintain it over years. Also it brings limitations, that sometimes can be workarounded in the virtio spec, but this always comes at a cost of decreased readability and increased complexity. Overall it looks clearly as a downgrade compared to virtio-video for our use-case. And I believe it would be the same for every developer, that has to actually implement the spec, not just do the pass through. So if we think of V4L2 UAPI pass through as a compatibility device (which I believe it is), then it is fine to have both and keep improving the virtio-video, including taking the best ideas from the V4L2 and overall using it as a reference to make writing the driver simpler.Let me jump in here and ask another question: Imagine that, some years in the future, somebody wants to add a virtio device for handling video encoding/decoding to their hypervisor. Option 1: There are different devices to chose from. How is the person implementing this supposed to pick a device? They might have a narrow use case, where it is clear which of the devices is the one that needs to be supported; but they also might have multiple, diverse use cases, and end up needing to implement all of the devices. Option 2: There is one device with various optional features. The person implementing this can start off with a certain subset of features depending on their expected use cases, and add to it later, if needed; but the upfront complexity might be too high for specialized use cases. Leaving concrete references to V4L2 out of the picture, we're currently trying to decide whether our future will be more like Option 1 or Option 2, with their respective trade-offs. I'm slightly biased towards Option 2; does it look feasible at all, or am I missing something essential here? (I had the impression that some previous confusion had been cleared up; apologies in advance if I'm misrepresenting things.) I'd really love to see some kind of consensus for 1.3, if at all possible :)I think feature discovery and extensibility is a key part of the VirtIO paradigm which is why I find the virtio-v4l approach limiting. By pegging the device to a Linux API we effectively limit the growth of the device specification to as fast as the Linux API changes. I'm not fully immersed in v4l but I don't think it is seeing any additional features developed for it and its limitations for camera are one of the reasons stuff is being pushed to userspace in solutions like libcamera: How is libcamera different from V4L2? We see libcamera as a continuation of V4L2. One that can more easily handle the recent advances in hardware design. As embedded cameras have developed, all of the complexity has been pushed on to the developers. With libcamera, all of that complexity is simplified and a single model is presented to application developers.Ok, that is interesting; thanks for the information.That said its not totally our experience to have virtio devices act as simple pipes for some higher level protocol. The virtio-gpu spec says very little about the details of how 3D devices work and simply offers an opaque pipe to push a (potentially propriety) command stream to the back end. As far as I'm aware the proposals for Vulkan and Wayland device support doesn't even offer a feature bit but simply changes the graphics stream type in the command packets. We could just offer a VIRTIO_VIDEO_F_V4L feature bit, document it as incompatible with other feature bits and make that the baseline implementation but it's not really in the spirit of what VirtIO is trying to achieve.I'd not be in favour of an incompatible feature flag, either... extensions are good, but conflicting features is something that I'd like to avoid. So, given that I'd still prefer to have a single device: How well does the proposed virtio-video device map to a Linux driver implementation that hooks into V4L2?
IMO it hooks into V4L2 pretty well. And I'm going to spend next few months making the existing driver fully V4L2 compliant. If this goal requires changing the spec, than we still have time to do that. I don't expect a lot of problems on this side. There might be problems with Android using V4L2 in weird ways. Well, let's see. Anyway, I think all of this can be accomplished over time.
If the general process flow is compatible and it is mostly a question of wiring the parts together, I think pushing that part of the complexity into the Linux driver is a reasonable trade-off. Being able to use an existing protocol is nice, but if that protocol is not perceived as flexible enough, it is probably not worth encoding it into a spec. (Similar considerations apply to hooking up the device in the hypervisor.)
I very much agree with these statements. I think this is how it should be: we start with a compact but usable device, then add features and enable them using feature flags. Eventually we can cover all the use-cases of V4L2 unless we decide to have separate devices for them (virtio-camera, etc). This would be better in the long term I think. If everybody wants a single device that much, I think it should be based on the current virtio-video. virtio-v4l2 spec can be developed out of tree for those, who need 100% compatibility right now. Maybe we can link it in the virtio-video, when it is ready?
Sorry about asking all those basic questions, but I really rely on the judgment of people familiar with the infrastructure and use cases so that we end up with a specification that is actually usable in the long term. Too many details are likely to confuse me :)
No worries. :) Thank you! -- Alexander Gordeev Senior Software Engineer OpenSynergy GmbH Rotherstr. 20, 10245 Berlin Phone: +49 30 60 98 54 0 - 88 Fax: +49 (30) 60 98 54 0 - 99 EMail: alexander.gordeev@opensynergy.com www.opensynergy.com Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B GeschÃftsfÃhrer/Managing Director: RÃgis Adjamah Please mind our privacy notice<https://www.opensynergy.com/datenschutzerklaerung/privacy-notice-for-business-partners-pursuant-to-article-13-of-the-general-data-protection-regulation-gdpr/> pursuant to Art. 13 GDPR. // Unsere Hinweise zum Datenschutz gem. Art. 13 DSGVO finden Sie hier.<https://www.opensynergy.com/de/datenschutzerklaerung/datenschutzhinweise-fuer-geschaeftspartner-gem-art-13-dsgvo/>
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]