[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [RFC PATCH v6] virtio-video: Add virtio video device specification
Hello, Sorry for the delay. I'm still gathering data in the old emails... On 08.12.22 08:23, Alexandre Courbot wrote:
Add the specification of the video decoder and encoder devices, which can be used to provide host-accelerated video operations to the guest. Signed-off-by: Keiichi Watanabe <keiichiw@chromium.org> Signed-off-by: Alexandre Courbot <acourbot@chromium.org> -- Here is the long-overdue new revision of the virtio-video RFC. This version reorganizes the specification quite a bit and tries to simplify the protocol further. Nonetheless, it still results in a rather long (17 pages) specification for just these devices, even though the spec is not fully complete (I want to rethink the formats descriptions, and some parameters need to be added for the encoder device). I would like to get some high-level feedback on this version and maybe propose to do things a bit differently before people invest too much time reviewing this in depth. While rewriting this document, it became more and more obvious that this is just a different, and maybe a bit simpler, reimplementation of the V4L2 stateless decoder protocol [1]. I am now wondering whether it would not make more sense to rewrite this specification as just a way to transport V4L2 requests over virtio, similarly to how virtio-fs does with the FUSE protocol [2]. At the time we started writing this implementation, the V4L2 protocols for decoders and encoders were not set in stone yet, but now that they are it might make sense to reconsider. Switching to this solution would greatly shorten the virtio-video device spec, and also provide a way to support other kind of V4L2 devices like cameras or image processors at no extra cost. Note that doing so would not require that either the host or guest uses V4L2 - the virtio video device would just emulate a V4L2 device over virtio. A few adaptations would need to be done regarding how memory types work, but otherwise I believe most of V4L2 could be used as-is. Please share your thoughts about this, and I will either explore this idea further with a prototype, or keep moving the present spec forward, hopefully at a faster pace. Due to the RFC state of this patch I have refrained from referencing the normative statements in conformance.tex - I will do that as a final step once the spec is mostly agreed on. [1] https://ddec1-0-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fdocs.kernel.org%2fuserspace%2dapi%2fmedia%2fv4l%2fdev%2dstateless%2ddecoder.html&umid=8ec8d8c9-b83c-40de-9337-a377056fe2af&auth=53c7c7de28b92dfd96e93d9dd61a23e634d2fbec-e98508782bc1c9aa6b2e4a9df9d4dd170f9a5ffa [2] https://github.com/oasis-tcs/virtio-spec/blob/master/virtio-fs.tex Full PDF: https://ddec1-0-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fdrive.google.com%2ffile%2fd%2f1HRVDiDdo50%2dS9X5tWgzmT90FJRHoB1dN%2fview%3fusp%3dsharing&umid=8ec8d8c9-b83c-40de-9337-a377056fe2af&auth=53c7c7de28b92dfd96e93d9dd61a23e634d2fbec-e315af79a067165e908bf1d803441eb181e2f375 PDF of video section only: https://ddec1-0-en-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2f%2fdrive.google.com%2ffile%2fd%2f1Sm6LSqvKqQiwYmDE9BXZ0po3XTKnKYlD%2fview%3fusp%3dsharing&umid=8ec8d8c9-b83c-40de-9337-a377056fe2af&auth=53c7c7de28b92dfd96e93d9dd61a23e634d2fbec-b5c45bb4b6ccc5b73ea2a54e26f151d61722d0df --- content.tex | 1 + virtio-video.tex | 1420 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 1421 insertions(+) create mode 100644 virtio-video.tex ...snip... +\subsection{Supported formats}\label{sec:Device Types / Video Device / Supported formats} + +Bitstream and image formats are identified by their fourcc code, which +is a four-bytes ASCII sequence uniquely identifying the format and its +properties. + +\subsubsection{Bitstream formats}\label{sec:Device Types / Video Device / Supported formats / Bitstream formats} + +The fourcc code of each supported bitstream format is given, as well as +the unit of data requested in each input resource for the decoder, or +produced in each output resource for the encoder. + +\begin{description} +\item[\field{MPG2}] +MPEG2 encoded stream. One Access Unit per resource. +\item[\field{H264}] +H.264 encoded stream. One NAL unit per resource. +\item[\field{HEVC}] +HEVC encoded stream. One NAL unit per resource. +\item[\field{VP80}] +VP8 encoded stream. One frame per resource. +\item[\field{VP90}] +VP9 encoded stream. One frame per resource. +\end{description} +
Actually I'm not sure where do these fourcc codes for bitstream formats come from. In one of the old comments I found a reference to https://www.rfc-editor.org/rfc/rfc2361 But it doesn't define MPG2, HEVC, VP80, VP90. Also there is this comment to virtio-video v1 from Tomasz Figa and the related discussion: https://markmail.org/message/gc6h25acct22niut#query:+page:1+mid:et4l3ni7qjqhiygo+state:results He wrote that it is not worth it because there are so many conflicting sets of fourcc codes. I can see that fourcc codes were not used in virtio-video spec draft versions 1 to 5. So now it looks quite weird to see them here. Probably this is because you'd like to mimic v4l2 interface more closely?
+\subsubsection{Image formats}\label{sec:Device Types / Video Device / Supported formats / Image formats} + +The fourcc code of each supported image format is given, as well as its +number of planes, physical buffers, and eventual subsampling. + +\begin{description} +\item[\field{RGB3}] +one RGB plane where each component takes one byte, i.e.~3 bytes per +pixel. +\item[\field{NV12}] +one Y plane followed by interleaved U and V data, in a single buffer. +4:2:0 subsampling. +\item[\field{NV12}] +same as \field{NV12} but using two separate buffers for the Y and UV +planes. +\item[\field{YU12}] +one Y plane followed by one Cb plane, followed by one Cr plane, in a +single buffer. 4:2:0 subsampling. +\item[\field{YM12}] +same as \field{YU12} but using three separate buffers for the Y, U and V +planes. +\end{description}
-- Alexander Gordeev Senior Software Engineer OpenSynergy GmbH Rotherstr. 20, 10245 Berlin Phone: +49 30 60 98 54 0 - 88 Fax: +49 (30) 60 98 54 0 - 99 EMail: alexander.gordeev@opensynergy.com www.opensynergy.com Handelsregister/Commercial Registry: Amtsgericht Charlottenburg, HRB 108616B GeschÃftsfÃhrer/Managing Director: RÃgis Adjamah Please mind our privacy notice<https://www.opensynergy.com/datenschutzerklaerung/privacy-notice-for-business-partners-pursuant-to-article-13-of-the-general-data-protection-regulation-gdpr/> pursuant to Art. 13 GDPR. // Unsere Hinweise zum Datenschutz gem. Art. 13 DSGVO finden Sie hier.<https://www.opensynergy.com/de/datenschutzerklaerung/datenschutzhinweise-fuer-geschaeftspartner-gem-art-13-dsgvo/>
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]