[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [RFC PATCH] Introduce admin virtqueue as a new transport
在 2021/8/4 下午9:36, Michael S. Tsirkin 写道:
On Tue, Aug 03, 2021 at 11:20:06AM +0800, Jason Wang wrote:This patch introduces a new transport - the admin virtqueue. This transport is useful for implementing virtual devices with a limited transport specific resources or presenting the virtual device in a transport independent way. This means, all the basic device facilities are provided solely via the the admin virtqueue. Additionally, the admin virtqueue is also in charge of the creating and destroying of the virtual device. To be self-contained and not depend on the platform specific feature. Device MMU is also introduced for providing the DMA isolation among virtual devices. With the help of the admin virtqueue, the presenting of the virtual device is done via the co-operation between the management device and its driver. This is just a draft for demonstrating the basic ideas. Some possible enhancements: - admin event virtqueue for reporting events like interrupts (on the platform withouth MSI) and MMU translation failure - hardware friendly MMU translation table (e.g in the memory instead of using control virtqueue commands) - command to kick the virtqueue Comments are more than welcomed. Signed-off-by: Jason Wang <jasowang@redhat.com> --- content.tex | 639 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 639 insertions(+) diff --git a/content.tex b/content.tex index 620c0e2..1f66d42 100644 --- a/content.tex +++ b/content.tex @@ -47,6 +47,9 @@ \section{\field{Device Status} Field}\label{sec:Basic Facilities of a Virtio Dev \item[DRIVER_OK (4)] Indicates that the driver is set up and ready to drive the device.+\item[DEVICE_MMU_FAIL (32)] Indicates that the device MMU has+ experienced an error from which it can't recover. + \item[DEVICE_NEEDS_RESET (64)] Indicates that the device has experienced an error from which it can't recover. \end{description} @@ -515,6 +518,642 @@ \chapter{Virtio Transport Options}\label{sec:Virtio Transport Options} Virtio can use various different buses, thus the standard is split into virtio general and bus-specific sections.+\section{Virtio Over Admin Virtqueue}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue}+ +Sometimes it's hard to implement the device in a transport specific +method. One example is that a physical device may try to present +multiple virtual devices with a limited transport specific +resources. Another example is to implement virtual devices which is +transport independent. In those cases, the admin virtqueue provided by +the management device could be used to replace the transport specific +method to implement the virtual device.terminology here needs clarification. Especially talking about virtual devices is confusing. I propose management device and managed device. Other options exist.
Sure, will do.
Also pls give examples such as PF/VF.
Ok.
Then the presenting of the +virtual device is done through the cooperation between the admin +virtqueue and the driver.A natural question to ask is why is this a VQ and not a device? Is this because people want to implement a VQ as part of an arbitrary device?
Yes, and actually, I don't want to exclude the possibility of having a dedicated management device.
But using an arbitrary device may have advantages (e.g for net it have the interface for remote management).
And based on the discussion with Stefan, I think we can add the following text:
"The management device MUST offer the admin virtqueue in a device specific virtqueue
"
+\subsection{Basic Concepts}\label{sec:Virtio Transport Options / Virtio over Admin Virtqueue / Basic Concepts} + +The device that offers the admin virtqueue (via feature +VIRTIO_F_ADMIN_VQ) is the management device of the virtual +devices.Don't we need a way to specify how many such VQs are there and what their numbers are?
Yes.
Doing this in a device specific way seems a bit annoying ...
I'm not sure I get the comment here, the above text is the definition of the management device.
All commands are of the following form: + +\begin{lstlisting} +struct virtio_admin_ctrl { + u64 device_id; + u16 class; + u16 command; + u8 command-out-data[]; + u8 ack; + u8 command-in-data[] +}; + +/* ack values */ +#define VIRTIO_ADMIN_OK 0 +#define VIRTIO_ADMIN_ERR 1 +\end{lstlisting} + +The device_id, class, command and command-out-data are set by +the driver, and the device sets the ack and command-in-data. 0 is used +for identify the management device itself. + +\devicenormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Admin Virtqueue / Basic Concepts} + +The virtual device MUST not offer VIRTIO_F_ADMIN_VQ feature. + +\drivernormative{\subsubsection}{Basic Concepts}{Virtio Transport Options / Virtio Over Admin Virtqueue / Basic Concepts} + +The driver SHOULD negotiate VIRTIO_F_ADMIN_VQ if the device offers it. + +\subsection{Virtual Device Discovery}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtual Device Discovery} + +The management device is discovered through a transport and device +specific method. Virtual devices is created and discovered via the +admin virtqueue. + +\subsection{Admin Virtqueue Capabilities}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Admin Virtqueue Capabilities} + +The capabilites that are supported by the admin virtqueue could be +fetched through the following commands: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_CAP 0 + #define VIRTIO_ADMIN_CTRL_CAP_GET 0 +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_CAP_GET is used to get the capabilites that are +supported by the admin virtqueue through a u64 which is a bit mask of +the capabilies in command-in-data. There's no command-out-data. + +The capabilies that is currently supported are: + +\begin{lstlisting} +#define VIRTIO_ADMIN_F_CAP_VDEV 1 +\end{lstlisting} + +The VIRTIO_ADMIN_F_CAP_VDEV capability demonstrates that the virtual +devices is created, configured and destroyed through admin +virtqueue. That means the admin virtqueue is the transport for the +virtual devices.How about using a feature bit for this? Or having this in the config space. This might call for a generic config space feature, but it's not the first time we want that, maybe it's time.
Feature bit works and seems better than the config space. I can change.
+\devicenormative{Admin Virtqueue Capabilities}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Admin Virtqueue Capabilities} + +The management device MUST support VIRTIO_ADMIN_CTRL_CAP class when +VIRTIO_F_ADMIN_VQ is offered. + +The management device MUST fail VIRTIO_ADMIN_CTRL_CAP class when the +\field{device_id} is not zero. + +\drivernormative{Admin Virtqueue Capabilities}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Admin Virtqueue Capabilities} + +The driver MUST use 0 as \field{device_id} for VIRTIO_ADMIN_CTRL_CAP +class. + +\subsection{Device Management}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Management} + +When the admin virtqueue offers VIRTIO_ADMIN_F_CAP_VDEV capibility, +virtual devices must be created and discovered through the admin +virtqueue. + +\begin{lstlisting} +struct virtio_admin_ctrl_vdev_attribute { + u32 device_id; + u8 config[]; +}; + +#define VIRTIO_ADMIN_CTRL_VDEV 2 + #define VIRTIO_ADMIN_CTRL_VDEV_CREATE 0 + #define VIRTIO_ADMIN_CTRL_VDEV_DESTROY 1 +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_VDEV_CREAT command is used to create a virtual +device. The command-out-data for VIRTIO_ADMIN_CTRL_CREATE is the +virtio device id (\field{device_id}) and device specific configuration +(\field{config}) for creating the virtual device. When succeed, the +device returns a u64 as a unique identifier of the created virtual +device in command-in-data.So how are we going to specify config? Per device type?
Yes, the config format is device specific. I will add some text for this.
+The VIRTIO_ADMIN_CTRL_VDEV_DESTROY command is used to destroy a +virtual device which is identified by its 64bit identifier +\field{virtual_device_id}. There's no command-in-data for +VIRTIO_ADMIN_CTRL_DESTROY command.So I am confused here. Rest of the spec seems to map driver actions to commands on the admin VQ. However where do the create and destroy commands coming from? If they have a separate source from driver commands, why do they share the same VQ?
It means the virtual/managed devices are provisioned dynamically. That is the hardware can structure the resources of virtual/managed device based on the requests from the user.
E.g when user is asking to create a networking device with 4 queue pairs. The hardware build up the resources dynamically during VDEV_CREAT and the resources are freed during VDEV_DESTROY.
I think we can add the support for statically pre-allocated virtual/managed device.
+\devicenormative{Device Management}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Management} + +The management device MUST fail the VIRTIO_ADMIN_CTRL_VDEV_CREATE if +\field{device_id} is not 0. + +The management device MUST fail the VIRTIO_ADMIN_CTRL_VDEV_DESTROY if +\field{device_id} is 0. + +All virtual devices MUST be created via admin virtqueue if the admin +virtqueue offers VIRTIO_F_CTRL_VDEV. + +The management device MAY map implement the virtual device in a +transport specific way.I'm not sure what does this mean.
Yes, will remove.
+\drivernormative{Driver Management}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Management} + +The management driver MUST use 0 as \field{device_id} for +VIRTIO_ADMIN_CTRL_VDEV_CREATE command. + +The management driver SHOULD make sure the virtual device is not used +by any driver before trying to destroy it.Device drivers are within guests. Not sure how this can be accomplished.
I can remove this. But even if drivers are within guests, form the host it's still bound to a driver.
+ +\subsection{Features Negotiation}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Features Negotiation} + +When the admin virtqueue offers VIRTIO_ADMIN_F_CAP_VDEV capability, +the feature negotiation of virtual devices could be done by the +following commands: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_FEAT 3 + #define VIRTIO_ADMIN_CTRL_FEAT_DEVICE_GET 0 + #define VIRTIO_ADMIN_CTRL_FEAT_DRIVER_SET 1 + #define VIRTIO_ADMIN_CTRL_FEAT_DRIVER_GET 2 +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_FEAT_DEVICE_GET is to get the features offered +by a virtual device. + +The VIRTIO_ADMIN_CTRL_FEAT_DRIVER_SET is for driver to accept feature +bits offered by the virtual device. + +The VIRTIO_ADMIN_CTRL_FEAT_DRIVER_GET is to get the features accepted +by both the virtual driver and the device.So there's a lot of text here to basically pass config read/writes over a VQ. How about specifying admin VQ in terms of e.g. virtio PCI transport? Thus basically supply read/write commands and that's it?
I just simply duplicate the functions of PCI transport. Maybe you can give an example of your idea? Is it something like read and write with an offset and define the commands as the offset?
+The features is 64 bits mask of the virtio features bit. For +VIRTIO_ADMIN_CTRL_DRIVER_SET, the feature is passed to the device +through command-out-data. For VIRTIO_ADMIN_CTRL_FEAT_DEVICE_GET and +VIRTIO_ADMIN_CTRL_DRIVER_GET the feature is returned for the device +through command-in-data. + +\devicenormative{Features Negotiation}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Features Negotiation} + +The management device MUST fail VIRTIO_ADMIN_F_CTRL_FEAT class for the +command that use 0 as its \field{virtual_device_id}. + +\drivernormative{Features Negotiation}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Features Negotiation} + +The management driver MAY mediate between the feature negotiation +request of the virtual devices and the admin virtqueue. E.g when +offering features to the virtual device, the management driver MAY +exclude some features in order to limit the behaviour of the virtual +device. + +\subsection{Device Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Status} + +When the admin virtqueue offers VIRTIO_ADNIN_F_CAP_VDEV capability, +the status of virtual device could be accessed by the following +commands: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_STATUS 4 + #define VIRTIO_ADMIN_CTRL_STATUS_SET 0 + #define VIRTIO_ADMIN_CTRL_STATUS_GET 1 +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_STATUS_SET is used to set the device status of +the virtual device here. The command-out-data is the one byte status +to set to the device. There's no command-in-data for this command. + +The VIRTIO_ADMIN_CTRL_STATUS_GET is used to get the device status of +the virtual device. The command-in-data is the one byte status +returned from the device. There's no command-out-data for this +command. + +\devicenormative{Device Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Status} + +The management device MUST start the reset of a virtual device when 0 +is written via VIRTIO_ADMIN_CTRL_STATUS_SET, the success of this +command demonstrate the success of the reset. + +The management device MUST present 0 through +VIRTIO_ADMIN_CTRL_STATUS_GET once the reset is done. + +The management device MUST fail the device status access if +\field{device_id} is zero. + +\drivernormative{Device Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Status} + +After writing 0 via VIRTIO_ADMIN_CTRL_STATUS_SET, the driver MUST wait +for the success of the command before re-initializing the device. + +\subsection{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Genreation} + +When the admin virtqueue offers VIRTIO_ADMIN_F_CAP_VDEV capability, +the device generation could be read from the following commands: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_GENERATION 5 + #define VIRTIO_ADMIN_CTRL_GENERATION_GET 0 +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_GENERATION_GET is used to get the device generation +of the virtual device. The command-in-data is the u32 device +generation returned from the device. There's no command-out-data for +this command. + +\devicenormative{Device Generation}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Generation} + +The device MUST present a changed config_generation after the driver +has read a device-specific configuration value which has changed since +any part of the device-specific configuration was last read. + +The device MUST fail the device generation access if \field{device_id} is zero. + +\subsection{Device Specific Configuration}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Specific Configuration} + +When the admin virtqueue offers VIRTIO_ADMIN_F_CAP_VDEV, the +config space of a virtual device could be accessed from +VIRTIO_ADMIN_CTRL_CONFIG_GET and VIRTIO_ADMIN_CTRL_CONFIG_SET. + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_CONFIG 6 + #define VIRTIO_ADMIN_CTRL_CONFIG_GET 0 + #define VIRTIO_ADMIN_CTRL_CONFIG_SET 1 + +struct virtio_admin_ctrl_vdev_config_get { + u32 offset; + u32 size; +}; + +struct virtio_admin_ctrl_vdev_config_set { + u32 offset; + u32 size; + u8 data[]; +}; +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_CONFIG_GET is used to read data from the +device configuration space. As described in struct +virtio_admin_ctrl_vdev_config_get, The command-out-data is the offset +since the start of the config space and the size of the data. The +command-in-data is the array of u8 data that read from the config +space. + +The VIRTIO_ADMIN_CTRL_CONFIG_SET is used to write data to the device +configuration space. As described in struct +virtio_admin_ctrl_vdev_config_set, the command-out-data contains the +offset since the start of the config space, the size of the data and +the data that will be wrote. There's no command-in-data for this +command. + +\devicenormative{Device Specific Configuration}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device Specific Configuration} + +The management device MUST fail the device configuration space access +if the driver want to access the range which is out of the config +space. + +The management device MUST fail the device configuration space access +if \field{device_id} is zero. + +\subsection{MSI Configuration}label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / MSI Configuration} + +When the admin virtqueue offers VIRTIO_ADMIN_F_VDEV, the MSI entry +for a specific virtqueue could be set through following command:I think this is a bit problematic. E.g. for PCI MSI is programmed through standard registers.
That's not conflict AFAIK. E.g if the management device is PCI, the standard registers will still work.But the virtual/managed device is not a PCI device, we need another way to associate MSI to them
Another important reason is that, we want to scale better than the PCI standard registers which only support about 2048 MSI vectors.
Specifying address is data is insufficient,
We have both addr and data and associate it with a specific virtqueue. I think it can work or anything I missed?
neither is masking and enabling through device specific registers. Referring to a vector seems more correct.
See above, we want to scale more than 2048, and the masking is per vq not per vector.
Or technically we can use vector if you wish, e.g have MSI-X table per virtual device.
Further, need to think about how will all this be generalized outside of PCI.
AFAIK, MSI is already used beyond the scope of PCI, the examples are some platform device and even the APIC interrupt. E.g in x86 a dedicated range of address space is used for MSI interrupt.
Thanks
+\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_MSI 7 + #define VIRTIO_ADMIN_CTRL_MSI_VQ_SET 0 + #define VIRTIO_ADMIN_CTRL_MSI_VQ_ENABLE 1 + #define VIRTIO_ADMIN_CTRL_MSI_VQ_MASK 2 + #define VIRTIO_ADMIN_CTRL_MSI_CONFIG_SET 3 + #define VIRTIO_ADMIN_CTRL_MSI_CONFIG_ENABLE 4 + #define VIRTIO_ADMIN_CTRL_MSI_CONFIG_MASK 5 + +struct virtio_admin_ctrl_vdev_msi_vq_set { + u16 queue_index; + u64 addr; + u32 data; +}; + +struct virtio_admin_ctrl_vdev_msi_vq_enable { + u16 queue_index; + u8 enable; +}; + +struct virtio_admin_ctrl_vdev_msi_vq_mask { + u16 queue_index; + u8 mask; +}; + +struct virtio_admin_ctrl_vdev_msi_config { + u64 addr; + u32 data; +}; +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_MSI_VQ_SET is used to set the MSI entry for a +specific virtqueue. The command-out-data is the virtqueue index and +the MSI address and data (as described in struct +virtio_admin_ctrl_vdev_msix_vq_set). + +The VIRTIO_ADMIN_CTRL_MSI_VQ_ENABLE and is used to enable or disable +MSI interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and whether to enable the MSI: 0 means to enable and 1 +means to disable (as described in struct +virtio_admin_ctrl_vdev_msi_vq_enable). + +The VIRTIO_ADMIN_CTRL_MSI_VQ_MASK and is used to mask or unmask MSI +interrupt for a specific virtqueue. The command-out-data is the +virtqueue index and the mask status: 0 means unmak and 1 means mask +(as described in struct virtio_admin_ctrl_vdev_msi_vq_mask). + +The VIRTIO_ADMIN_CTRL_MSI_CONFIG_SET is used to set the MSI entry +for the config interrupt. The command-out-data is the MSI address and +data (as described in struct virtio_admin_ctrl_vdev_msix_config). + +The VIRTIO_ADMIN_CTRL_MSI_CONFIG_ENABLE is used to enable and disable +MSI for config space. The command-out-data is an u8: 0 means to +disable and 1 means to enable. + +The VIRTIO_ADMIN_CTRL_MSI_CONFIG_MASK is used to mask and unmask MSI +interrupt for config space. The command-out-data is an u8: 0 means to +mask and 1 means to unmask. + +There's no command-in-data for all the above MSI commands. + +\devicenormative{MSI Configuration}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / MSI Configuration} + +The virtual device MUST record the pending MSI interrupt and +generate the MSI interrupt is it was pending after unmasking. + +The virtual MUST disable the MSI for both virtqueue and config space +upon reset. + +\drivernormative{MSI Configuration}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / MSI Configuration} + +The driver MUST allocate transport or platform specific MSI entries +for both virtqueue and config space if it wants to use interrupt. + +The driver MAY choose disable the MSI if polling is used. + +\subsection{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Address} + +When the admin virtqueue offers VIRTIO_ADMIN_F_CAP_VDEV, the address of +a specific virtqueue could be done through the following command: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_VQ_ADDR 9 + #define VIRTIO_ADMIN_CTRL_VQ_ADDR_SET 0 + +struct virtio_admin_ctrl_vdev_vq_addr { + u16 queue_index; + u64 device_area; + u64 descriptor_area; + u64 driver_area; +}; +\end{lstlisting} + +The command-out-data is the queue index, the addresses of device area, +descriptor area and driver area (as described in struct +virtio_admin_ctrl_vdev_vq_addr); There's no command-in-data. + +\devicenormative{Virtqueue Address}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueeu Address} + +The management device MUST fail the commands of class +VIRTIO_ADMIN_CTRL_VQ_ADDR if \field{device_id} is zero. + +\subsection{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Status} + +When the admin virtqueue offers VIRTIO_F_ADMIN_F_CAP_VDEV, virtqueue +status could be set and get through the following command: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_VQ_ENABLE 10 + #define VIRTIO_ADMIN_CTRL_VQ_ENABLE_SET 0 + #define VIRTIO_ADMIN_CTRL_VQ_ENABLE_GET 1 + +struct virtio_admin_ctrl_vq_status_set { + u16 queue_index; + u8 status; +}; + +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_VQ_ENABLE_SET is used to set the status to a +specific virtqueue. The command-out-data is the queue index, the +status that is set to the virtqueue (0 disabled, 1 enabled); There's +no command-in-data. + +The VIRTIO_ADMIN_CTRL_VQ_ENABLE_GET is used to get the status of a +specific virtqueue. The command-out-data is the u16 of queue +index. The command-in-data is the virtqueue status (0 disalbed, 1 +enabled). + +\devicenormative{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Status} + +When disabled, the virtual device MUST stop processing requests from +this virtqueue. + +The management device MUST present a 0 via +VIRTIO_ADMIN_CTRL_VQ_ENABLE_GET on reset of the virtual device. + +The management device MUST fail the virtqueue status access if +\field{device_id} is zero. + +\devicenormative{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Status} + +The driver MUST configure the other virtqueue fields before enabling +the virtqueue with VIRTIO_ADMIN_CTRL_VQ_ENABLE_SET. + +\subsection{Virtqueue Size}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Size} + +When the admin virtqueue offers VIRTIO_ADMIN_F_VDEV, virtqueue size +could be accessed through the following command: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_VQ_SIZE 11 + #define VIRTIO_ADMIN_CTRL_VQ_SIZE_SET 0 + #define VIRTIO_ADMIN_CTRL_VQ_SIZE_GET 1 + +struct virtio_admin_ctrl_vdev_vq_size_set { + u16 queue_index; + u16 size; +}; +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_VQ_SIZE_SET command is used to set the virtqueue +size. The command-out-data is the queue index and the size of the +virtqueue (as described in struct +virtio_admin_ctrl_vdev_vq_size_set). There's no command-in-data. + +The VIRTIO_ADMIN_CTRL_VQ_SIZE_GET command is used to get the virtqueue +size. On reset, the maximum queue size supported by the device is +returned. The command-out-data is the u16 of the virtqueue index. The +command-in-data is the u16 of queue size for the virtqueue. + +\devicenormative{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Size} + +The management device MUST fail the virtqueue size access if +\field{device_id} is zero. + +\subsection{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Notification} + +When the admin virtqueue offers VIRTIO_ADMIN_F_VDEV, the virtqueue +notification could be done through the following commands: + +\begin{lstlisting} +#define VIRTIO_ADMIN_CTRL_VQ_NOTIFY 12 + #define VIRTIO_ADMIN_CTRL_VQ_NOTIFY_GET 1 + +struct virtio_admin_ctrl_vdev_vq_notification_area { + le64 addr + le64 size; +}; +\end{lstlisting} + +The VIRTIO_ADMIN_CTRL_VQ_NOTIFY_GET is used to get the transport +specific address area that can be used to notify a virtqueue. The +command-out-data is a u16 of the virtqueue index. The command-in-data +contains the address and the size of the notification area (as +described in struct virtio_admin_ctrl_vdev_vq_notification_area). + +\devicenormative{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Notification} + +The management device MUST fail the VIRTIO_ADMIN_CTRL_VQ_NOTIFY_GET if +there's no transport specific notification address for a virtqueue of +its virtual device. + +The management device MUST fail the virtqueue notification access if +\field{device_id} is zero. + +The management device MUST forbid the notification area of a specific +virtual device to be accessed from another virtual device. + +\drivernormative{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Virtqueue Notification} + +The driver MAY choose to notify the virtqueue by writing the queue +index at address \field{addr} which is fetched from the +VIRTIO_ADMIN_CTRL_VQ_NOTIFY_GET command. + +\subsection{Virtqueue Notification}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device MMU} + +When the admin virtqueue offers VIRTIO_ADMIN_F_VDEV, management device +offers a device MMU for a secure DMA context for each virtual +device. The device MMU will translate I/O Virtual Address to transport +specific DMA address before using a transport specific way for DMA: + +\begin{lstlisting} +#define VIRTIO_ADMIN_VQ_CTRL_MMU 13 + #define VIRTIO_ADMIN_VQ_CTRL_MMU_ENABLE 1 + #define VIRTIO_ADMIN_VQ_CTRL_MMU_ASID_SET 2 + #define VIRTIO_ADMIN_VQ_CTRL_MMU_MAP 3 + #define VIRTIO_ADMIN_VQ_CTRL_MMU_UNMAP 4 + #define VIRTIO_ADMIN_VQ_CTRL_MMU_ERR_GET 5 + +struct virtio_admin_ctrl_vdev_mmu_asid_set { + le16 queue_index; + le64 asid; +}; + +struct virtio_admin_ctrl_vdev_mmu_map { + le64 iova_start; + le64 iova_end; + le64 dma_start; + le32 flags; +}; + +/* Read access is allowed */ +#define VIRTIO_ADMIN_VQ_MAP_F_READ (1 << 0) +/* Write access is allowed */ +#define VIRTIO_IOMMU_VQ_MAP_F_WRITE (1 << 1) + +struct virtio_admin_ctrl_vdev_mmu_err { + le32 reason; + le16 queue_index; + le64 asid; + le64 iova_start; + le64 iova_end; + le32 flags; +}; + +/* Mapping does not exist */ +#define VIRTIO_ADMIN_VQ_MAP_ERR_NON_EXIST (1 << 0) +/* Access violates the permission */ +#define VIRTIO_ADMIN_VQ_MAP_ERR_ACC_VIOLATION (1 << 1) + +\end{lstlisting} + +The VIRTIO_ADMIN_VQ_CTRL_MMU_ENABLE is used to enable or disable +device MMU for a specific virtual device. The command-out-data is a u8 +for telling whether device MMU is enabled for the virtual device: 0 +means to enable and 1 means to disable. + +The VIRTIO_ADMIN_VQ_CTRL_MMU_ASID_SET is used to assign a device +address space id to a virtqueue. The command-out-data is the queue +index (\field{queue_index}) and the address space ID (\field{asid}) +assigned to it (as described in struct virtio_admin_ctrl_vdev_mmu_asid_set). + +The VIRTIO_ADMIN_VQ_CTRL_MMU_MAP is used to map the I/O Virtual +Address range [\field{iova_start}, \field{iova_end}] to transport +specific DMA address range [\field{dma_start}, \field{dma_start} + + \field{iova_end} - \field{iova_start} + 1]. \field{flags} is used to +specify the device access permission. + +The VIRTIO_ADMIN_VQ_CTRL_MMU_UNMAP is used to unmap all the mapped I/O +Virtual Address ranges that are intersected with the range +[\field{iova_start}, \field{iova_end}]. + +There's no command-in-data for all the above four commands. + +The VIRTIO_ADMIN_VQ_CTRL_MMU_ERR_GET is used to get the error +information of the device MMU. There's no command-out-data, the +command-in-date is the queue index and its asid, the iova range and +the access of the operation (as described in struct +virtio_admin_ctrl_vdev_mmu_err). + +\devicenormative{Virtqueue Status}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Device MMU} + +The management device MUST fail the device MMU command if \field{device_id} is +zero. + +The management device MUST fail the VIRTIO_ADMIN_VQ_CTRL_MMU_MAP +command if the iova range is intersected with a existing range. + +The management device MUST set both DEVICE_NEEDS_RESET and +DEVICE_MMU_FAIL when the device MMU fails to do the translation for a +virtual device. + +The device MMU for the virtual device MUST be disabled upon its reset. + +Upon reset, the virtual device must reset the Address Space ID for +each virtqueue to 0. + +\drivernormative{Virtqueue Status}\label{sec:Virtio Transport Options + / Virtio Over Admin Virtqueue / Device MMU} + +The driver MAY choose to disable the device MMU but it MUST make sure +the transport specific method could be used to provide a secure DMA +context for each virtual device. + +The driver MAY query the error of device MMU after DEVICE_MMU_FAIL is set. + +\subsection{Presenting Virtual Device}\label{sec:Virtio Transport Options / Virtio Over Admin Virtqueue / Presenting Virtual Device} + +If VIRTIO_ADMIN_F_VDEV is offered by the device. The presenting of +the virtual device requires co-operation between the management +driver and the admin virtqueue. This means, from the view of the +virtual device driver, the transport is done via the communication +with the management device driver. It's up to the software to decide +what kind of method that is needed be used for those communications. + +The management driver typically do the following steps for creating a +virtual device: + +\begin{enumerate} +\item Determine the virtio id and device specific configuration. +\item Create the virtual devices using VIRTIO_ADMIN_CTRL_VDEV_CREATE +command. +\item Optionally, configure the MSI. +\item Optionally, enable and initialize the device MMU. +\item Setup the necessary communication methods with virtual device driver. +\item Perform device specific setups. +\item Let the virtual device to be probed by the virtual device +driver. The management driver will then use the admin virtqueue to +implement the requests of basic facility from the virtual device +driver. +\end{enumerate} + \section{Virtio Over PCI Bus}\label{sec:Virtio Transport Options / Virtio Over PCI Bus}Virtio devices are commonly implemented as PCI devices.-- 2.24.3 (Apple Git-128)
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]