[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [PATCH v2 1/9] virtio-iommu: Add ATTACH_TABLE request
Resend the reply and CC virtio-comment list. Hi, On 10/7/23 00:09, Jean-Philippe Brucker wrote:
This is what VT-d hardware expects when working with viommu (no matter it's virtual VT-d iommu or virtual-iommu) with VT-d IO page table support. If a device is directly assigned to a guest, the guest viommu driver fully manages the PASIDs of that device. If a device is providing I/O device sharing service for multiple guests, the PASIDs value are assigned by host. But, in either case, the PASID table is filled by host VT-d driver, not guest viommu driver. So, from viommu front-end's point of view, the PASID table of that device is managed by viommu device (i.e., viommu back-end).Add a way to attach page tables to a domain. Only introduces the feature bit and the ATTACH_TABLE request. We'll also need new PROBE properties and INVALIDATE requests. It could also be ATTACH followed by "SET_TABLE", or "ALLOC_WITH_TABLE" followed by ATTACH, but this leads to fewer domain states. With ATTACH_TABLE a domain can be free, attached-with-map or attached-with-pgtable. Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org> --- device-types/iommu/description.tex | 113 +++++++++++++++++++++++++---- 1 file changed, 100 insertions(+), 13 deletions(-) diff --git a/device-types/iommu/description.tex b/device-types/iommu/description.tex index d9d484c..f81d96d 100644 --- a/device-types/iommu/description.tex +++ b/device-types/iommu/description.tex @@ -54,9 +54,7 @@ \subsection{Feature bits}\label{sec:Device Types / IOMMU Device / Feature bits} The number of domains supported is described in \field{domain_range}.\item[VIRTIO_IOMMU_F_MAP_UNMAP (2)]- Map and unmap requests are available.\footnote{Future extensions may add - different modes of operations. At the moment, only - VIRTIO_IOMMU_F_MAP_UNMAP is supported.} + Map and unmap requests are available.\item[VIRTIO_IOMMU_F_BYPASS (3)]Endpoints that are not attached to a domain are in bypass mode. @@ -74,6 +72,8 @@ \subsection{Feature bits}\label{sec:Device Types / IOMMU Device / Feature bits} whether endpoints that are attached to a domain are in bypass mode.+\item[VIRTIO_IOMMU_F_ATTACH_TABLE (7)]+ The ATTACH_TABLE and INVALIDATE requests are available. \end{description}\drivernormative{\subsubsection}{Feature bits}{Device Types / IOMMU Device / Feature bits}@@ -84,7 +84,8 @@ \subsection{Feature bits}\label{sec:Device Types / IOMMU Device / Feature bits}\devicenormative{\subsubsection}{Feature bits}{Device Types / IOMMU Device / Feature bits} -The device SHOULD offer feature bit VIRTIO_IOMMU_F_MAP_UNMAP.+The device MUST offer at least one of the +VIRTIO_IOMMU_F_MAP_UNMAP or VIRTIO_IOMMU_F_ATTACH_TABLE features.The VIRTIO_IOMMU_F_BYPASS_CONFIG feature supersedesVIRTIO_IOMMU_F_BYPASS. New devices SHOULD NOT offer @@ -142,10 +143,6 @@ \subsection{Device initialization}\label{sec:Device Types / IOMMU Device / DevicWhen the device is reset, endpoints are not attached to any domain. -Future devices might support more modes of operation besides MAP/UNMAP.-Drivers verify that devices set VIRTIO_IOMMU_F_MAP_UNMAP and fail -gracefully if they don't. - \drivernormative{\subsubsection}{Device Initialization}{Device Types / IOMMU Device / Device Initialization}The driver MUST NOT negotiate VIRTIO_IOMMU_F_MAP_UNMAP if it is incapable@@ -177,11 +174,12 @@ \subsection{Device operations}\label{sec:Device Types / IOMMU Device / Device op Type may be one of:\begin{lstlisting}-#define VIRTIO_IOMMU_T_ATTACH 1 -#define VIRTIO_IOMMU_T_DETACH 2 -#define VIRTIO_IOMMU_T_MAP 3 -#define VIRTIO_IOMMU_T_UNMAP 4 -#define VIRTIO_IOMMU_T_PROBE 5 +#define VIRTIO_IOMMU_T_ATTACH 1 +#define VIRTIO_IOMMU_T_DETACH 2 +#define VIRTIO_IOMMU_T_MAP 3 +#define VIRTIO_IOMMU_T_UNMAP 4 +#define VIRTIO_IOMMU_T_PROBE 5 +#define VIRTIO_IOMMU_T_ATTACH_TABLE 6 \end{lstlisting}A few general-purpose status codes are defined here.@@ -395,6 +393,89 @@ \subsubsection{ATTACH request}\label{sec:Device Types / IOMMU Device / Device op attached to the domain, the device SHOULD reject the request and set \field{status} to VIRTIO_IOMMU_S_UNSUPP.+\subsubsection{ATTACH_TABLE request}\label{ref:Device Types / IOMMU Device / Device operations / ATTACH_TABLE request}+ +\begin{lstlisting} +struct virtio_iommu_req_attach_table { + struct virtio_iommu_req_head head; + le32 domain; + le32 endpoint; + u8 format; + u8 descriptor[111]; + struct virtio_iommu_req_tail tail; +}; +\end{lstlisting} + +Attach an endpoint to a domain, in the same way as an ATTACH +request. In addition, provide the address of a table describing +the mappings. Instead of using MAP and UNMAP requests, the driver +creates and removes mappings by writing into the tables. When a +mapping that may have been cached in a TLB is removed, the driver +sends an INVALIDATE request. + +The driver discovers with a PROBE request the table formats +supported by the device for each endpoint. The driver chooses a +format it recognises, and sends the ATTACH_TABLE request. + +Apart from \field{domain}, \field{endpoint} and \field{format}, +each table format uses different parameters in field +\field{descriptor}. Table formats and their associated parameters +are described in section \ref{sec:Device Types / IOMMU Device / +Acceleration}. + +Some formats support \emph{page tables}, where the ATTACH_TABLE +request contains the address of a \emph{page directory}, the root +of a page table tree which describe mappings of a single address +space. Other formats support \emph{PASID tables}, which contain +multiple page directory addresses, indexed by a 20-bit identifier +called PASID (Process Address Space ID), providing multiple +address spaces per endpoint. Field \field{domain} corresponds to +one PASID table: a second ATTACH_TABLE request with the same +\field{domain} and a different \field{endpoint} has the PASID +table address. The device is not required to support attaching +multiple endpoints to the same domain. + +For some formats, the PASID table is managed by the device rather +than driver. The ATTACH_TABLE request contains a \field{pasid}
+field and a page directory. Similarly to when the driver manages
+the PASID table, \field{domain} corresponds to one PASID table.
+The driver uses the same \field{domain} for each ATTACH_TABLE
+request with PASID. Issuing a DETACH request without PASID
>Here are some thoughts about domain id interception with VT-d IO page format:
Virtual VT-d driver uses the same domain id for first-stage translation is because it's required by the VT-d specification, though guest domain id never gets to hardware (i.e., host doesn't populate PASID table with guest domain id value).
According to VT-d specification (section 6.2.3.1): "Scalable-mode PASID table entries programmed for first-stage translation or pass-through (PGTT equal to 001b or 100b) must be programmed with a DID value that is different from those used in any PASID table entries that are programmed for second-stage or nested translation (PGTT equal to 010b or 011b). This is required since hardware implementations tag various caches with domain-id as described in Section 6.2.1. Scalable-mode PASID-table entries with PGTT value of 001b or 100b are recommended to use the same domain-id value for best hardware efficiency."
When VT-d IO page table comes to virtual-iommu, the definition of domain id doesn't need to be exactly the same with the one defined in VT-d specification as the viommu providing the service is virtual-iommu, not virtual VT-d and only VT-d IO page table is being used (domain id is defined in PASID table which isn't being used).
So, it seems more reasonable to define the domain id as an identifier for an unique domain for each address space for virtual-iommu back-end interception. Any ideas?
Regards, -Tina
+detaches all address spaces that were attached with PASID for the +given \field{domain}. + +An endpoint that supports issuing DMA with PASID can also issue +DMA without PASID. The configuration method of the address space +without PASID depends on the table format, but typically consists +in reserving the first entry of the PASID table for DMA without +PASID. In this case, for a format where the device manages PASID +tables, ATTACH_TABLE requests with \field{pasid} 0 attaches a +page directory for DMA without PASID, and DETACH request with +\field{pasid} 0 detaches it. A DETACH request without +VIRTIO_IOMMU_DETACH_F_PASID, on the other hand, detaches all page +directories for the given \field{domain}. + +\drivernormative{\paragraph}{ATTACH_TABLE request}{Device Types / IOMMU Device / Device operations / ATTACH_TABLE request} + +The driver SHOULD NOT send ATTACH_TABLE requests if the +VIRTIO_IOMMU_F_ATTACH_TABLE feature was not negotiated. + +The driver SHOULD NOT send ATTACH_TABLE requests with a +\field{domain} currently attached without table. It SHOULD NOT +send ATTACH requests with a \field{domain} currently attached +with table. + +\devicenormative{\paragraph}{ATTACH_TABLE request}{Device Types / IOMMU Device / Device operations / ATTACH_TABLE request} + +If the endpoint identified by \field{endpoint} is already +attached to another domain, then the device SHOULD first detach +it from that domain and attach it to the one identified by +\field{domain}. In that case the device SHOULD behave as if the +driver issued a DETACH request with this \field{endpoint}, +followed by the ATTACH_TABLE request. If the device cannot do so, +it MUST reject the request and set \field{status} to +VIRTIO_IOMMU_S_UNSUPP. + \subsubsection{DETACH request}\begin{lstlisting}@@ -512,6 +593,9 @@ \subsubsection{MAP request}\label{sec:Device Types / IOMMU Device / Device opera If the VIRTIO_IOMMU_F_MMIO feature isn't negotiated, the driver MUST NOT use the VIRTIO_IOMMU_MAP_F_MMIO flag.+\field{domain} SHOULD NOT have been created with an ATTACH_TABLE+request. + \devicenormative{\paragraph}{MAP request}{Device Types / IOMMU Device / Device operations / MAP request}If \field{virt_start}, \field{phys_start} or (\field{virt_end} + 1) is@@ -609,6 +693,9 @@ \subsubsection{UNMAP request}\label{sec:Device Types / IOMMU Device / Device opeThe driver SHOULD NOT send UNMAP requests on a bypass domain. +\field{domain} SHOULD NOT have been created with an ATTACH_TABLE+request. + \devicenormative{\paragraph}{UNMAP request}{Device Types / IOMMU Device / Device operations / UNMAP request}If the \field{reserved} field of an UNMAP request is not zero, the device
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]