Re: [virtio-dev] [PATCH RFC v4 0/1] Virtio Video Device Specification

[Keiichi Watanabe <keiichiw@chromium.org>]

On Fri, Jan 15, 2021 at 3:18 AM Alex BennÃe <alex.bennee@linaro.org> wrote:

Keiichi Watanabe <keiichiw@chromium.org> writes:

> This is the v4 RFC of virtio video device specification.
> PDF versions are available at [1, 2].
>
> Note that this patch depends on a recent patchset "Cross-device resource
> sharing" [3].
>
> Here is a list of major changes from v3:
> * Redesingned struct definitions for each command and request based on
>Â Âdiscussions at [4].
> * Renamed commands/structs/enums to more descriptive names.
> * Had different structs and fields for image formats and bitstream formats.
> * Added more detailed specification for supported video codecs.
> * Made stream_id be allocated by the device.
> * Had a single parameter struct per-stream instead of per-queue parameters and
>Â Âcontrols.
> * Allowed the driver to specify the number of buffers to use via
>Â Â"cur_{image,bitstream}_buffers".
> * Renamed RESOURCE_CREATE command to RESOURCE_ATTACH command and allow the
>Â Âdriver to use this command when replacing backing memories as well.
>
> [5] is the diff of the header file from v3. Note that it only contains changes
> in the header. We haven't updated the driver nor device implementation to focus
> on protocol design discussion first.
>
> While it may appear that many parts have been changed since the previous
> revision, these changes are to address the issues raised in previous discussions
> or/and to make the protocol simpler and easier to prevent misuse.
> I'd appreciate any types of feedback.
>
> Best regards,
> Keiichi
>
> [1] (full): https://drive.google.com/file/d/1DiOJZfUJ5wvFtnNFQicxt0zkp4Ob1o9C/view?usp=sharing
> [2] (only video section): https://drive.google.com/file/d/188uAkIWE0BsXETECez98y5fJKw8rslj3/view?usp=sharing
> [3] https://lists.oasis-open.org/archives/virtio-comment/202003/msg00035.html
> [4] https://markmail.org/thread/c6h3e3zn647qli3w
> [5]
> https://chromium-review.googlesource.com/c/chromiumos/third_party/kernel/+/2164411

Hi Keiichi,

I wanted to ask what the current status of this spec was. Are you
planning to submit a new revision of the specification due or are things
fairly stable now?

We are starting to think about next steps for virtualised video as part
of Linaro's Stratos work. Specifically we are thinking about
implementing backends and getting a stack up and running which we can
use to experiment with multiple hypervisors and VM deployment
approaches.

Longer term goals included looking at how to integrate virtio-video with
a secure world on ARM (e.g. feed video data to a secure world device for
playback via virtio). As part of that we will also be looking at how to
minimise the memory profile of the backend to do this.

Looking at the virtio-spec repo it looks like the cross-device resource
sharing is now merged:

 87fa6b5 * virtio-gpu: add support for mapping/unmapping blob resources
 89e7eb5 * virtio-gpu: add resource create blob
 162578b * virtio-gpu: add the ability to export resources
 68f66ff * content: define what an exported object is

are there any other prerequisites?

From a backend implementation point of view it makes sense to wait until
there is a working frontend driver up-streamed into the kernel. I guess
that is blocked on the final call for vote on the virtio spec?

I'm sure there is scope for parallelism here but I wanted to get a sense
of the current direction before embarking on work that would require a
big re-write down the line.

Hi Alex,

Thanks for reaching out to me. I'm really happy to hear your team is looking at virtio-video.

For the specification, we've been preparing the v5 spec by addressing previous review comments and simplifying rules. I think we'll be able to share it for reviewing soon. It won't require any additional mechanism other than the resource sharing feature.

For the implementation, unfortunately, we haven't started implementing a driver using the v5 protocol yet.

While we have the driver and device using the v3 protocol in Chrome OS repository, some efforts are required to update them to support the v5 protocol because there are some gaps between the v3 protocol and the v4/v5 protocol.

I think the driver implementation is necessary for the spec to be merged, but it's not yet clear when we can spend time implementing drivers. It's likely to be after April or so.

IIRC, OpenSynergy folks, who implemented the v3 driver, also had some plan to implement the driver with the v5 spec.

Matti, do you have any update on it? I'd really appreciate it if we could keep working for upstream together.

Regards,

Keiichi

Â

Regards,

--
Alex BennÃe