[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: Enabling hypervisor agnosticism for VirtIO backends
Hi Christopher,
Thank you for your feedback.
On Mon, Aug 30, 2021 at 12:53:00PM -0700, Christopher Clark wrote:
> [ resending message to ensure delivery to the CCd mailing lists
> post-subscription ]
>
> Apologies for being late to this thread, but I hope to be able to
> contribute to
> this discussion in a meaningful way. I am grateful for the level of
> interest in
> this topic. I would like to draw your attention to Argo as a suitable
> technology for development of VirtIO's hypervisor-agnostic interfaces.
>
> * Argo is an interdomain communication mechanism in Xen (on x86 and Arm)
> that
>Â Âcan send and receive hypervisor-mediated notifications and messages
> between
>Â Âdomains (VMs). [1] The hypervisor can enforce Mandatory Access Control
> over
>Â Âall communication between domains. It is derived from the earlier v4v,
> which
>Â Âhas been deployed on millions of machines with the HP/Bromium uXen
> hypervisor
>Â Âand with OpenXT.
>
> * Argo has a simple interface with a small number of operations that was
>Â Âdesigned for ease of integration into OS primitives on both Linux
> (sockets)
>Â Âand Windows (ReadFile/WriteFile) [2].
>Â Â Â- A unikernel example of using it has also been developed for XTF. [3]
>
> * There has been recent discussion and support in the Xen community for
> making
>Â Ârevisions to the Argo interface to make it hypervisor-agnostic, and
> support
>Â Âimplementations of Argo on other hypervisors. This will enable a single
>Â Âinterface for an OS kernel binary to use for inter-VM communication that
> will
>Â Âwork on multiple hypervisors -- this applies equally to both backends and
>Â Âfrontend implementations. [4]
Regarding virtio-over-Argo, let me ask a few questions:
(In figure "Virtual device buffer access:Virtio+Argo" in [4])
1) How the configuration is managed?
 ÂOn either virtio-mmio or virtio-pci, there always takes place
 Âsome negotiation between the FE and BE through the "configuration"
 Âspace. How can this be done in virtio-over-Argo?
2) Do there physically exist virtio's available/used vrings as well as
 Âdescriptors, or are they virtually emulated over Argo (rings)?
3) The payload in a request will be copied into the receiver's Argo ring.
 ÂWhat does the address in a descriptor mean?
 ÂAddress/offset in a ring buffer?
4) Estimate of performance or latency?
 ÂIt appears that, on FE side, at least three hypervisor calls (and data
 Âcopying) need to be invoked at every request, right?
Thanks,
-Takahiro Akashi
> * Here are the design documents for building VirtIO-over-Argo, to support a
>Â Âhypervisor-agnostic frontend VirtIO transport driver using Argo.
>
> The Development Plan to build VirtIO virtual device support over Argo
> transport:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
>
> A design for using VirtIO over Argo, describing how VirtIO data structures
> and communication is handled over the Argo transport:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1348763698/VirtIO+Argo
>
> Diagram (from the above document) showing how VirtIO rings are synchronized
> between domains without using shared memory:
> https://openxt.atlassian.net/46e1c93b-2b87-4cb2-951e-abd4377a1194#media-blob-url="">
>
> Please note that the above design documents show that the existing VirtIO
> device drivers, and both vring and virtqueue data structures can be
> preserved
> while interdomain communication can be performed with no shared memory
> required
> for most drivers; (the exceptions where further design is required are those
> such as virtual framebuffer devices where shared memory regions are
> intentionally
> added to the communication structure beyond the vrings and virtqueues).
>
> An analysis of VirtIO and Argo, informing the design:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1333428225/Analysis+of+Argo+as+a+transport+medium+for+VirtIO
>
> * Argo can be used for a communication path for configuration between the
> backend
>Â Âand the toolstack, avoiding the need for a dependency on XenStore, which
> is an
>Â Âadvantage for any hypervisor-agnostic design. It is also amenable to a
> notification
>Â Âmechanism that is not based on Xen event channels.
>
> * Argo does not use or require shared memory between VMs and provides an
> alternative
>Â Âto the use of foreign shared memory mappings. It avoids some of the
> complexities
>Â Âinvolved with using grants (eg. XSA-300).
>
> * Argo supports Mandatory Access Control by the hypervisor, satisfying a
> common
>Â Âcertification requirement.
>
> * The Argo headers are BSD-licensed and the Xen hypervisor implementation
> is GPLv2 but
>Â Âaccessible via the hypercall interface. The licensing should not present
> an obstacle
>Â Âto adoption of Argo in guest software or implementation by other
> hypervisors.
>
> * Since the interface that Argo presents to a guest VM is similar to DMA, a
> VirtIO-Argo
>Â Âfrontend transport driver should be able to operate with a physical
> VirtIO-enabled
>Â Âsmart-NIC if the toolstack and an Argo-aware backend provide support.
>
> The next Xen Community Call is next week and I would be happy to answer
> questions
> about Argo and on this topic. I will also be following this thread.
>
> Christopher
> (Argo maintainer, Xen Community)
>
> --------------------------------------------------------------------------------
> [1]
> An introduction to Argo:
> https://static.sched.com/hosted_files/xensummit19/92/Argo%20and%20HMX%20-%20OpenXT%20-%20Christopher%20Clark%20-%20Xen%20Summit%202019.pdf
> https://www.youtube.com/watch?v=cnC0Tg3jqJQ
> Xen Wiki page for Argo:
> https://wiki.xenproject.org/wiki/Argo:_Hypervisor-Mediated_Exchange_(HMX)_for_Xen
>
> [2]
> OpenXT Linux Argo driver and userspace library:
> https://github.com/openxt/linux-xen-argo
>
> Windows V4V at OpenXT wiki:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/14844007/V4V
> Windows v4v driver source:
> https://github.com/OpenXT/xc-windows/tree/master/xenv4v
>
> HP/Bromium uXen V4V driver:
> https://github.com/uxen-virt/uxen/tree/ascara/windows/uxenv4vlib
>
> [3]
> v2 of the Argo test unikernel for XTF:
> https://lists.xenproject.org/archives/html/xen-devel/2021-01/msg02234.html
>
> [4]
> Argo HMX Transport for VirtIO meeting minutes:
> https://lists.xenproject.org/archives/html/xen-devel/2021-02/msg01422.html
>
> VirtIO-Argo Development wiki page:
> https://openxt.atlassian.net/wiki/spaces/DC/pages/1696169985/VirtIO-Argo+Development+Phase+1
>
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]