[PATCH v2 0/9] virtio-iommu: Support page tables

[Jean-Philippe Brucker <jean-philippe@linaro.org>]

Add page table support to the virtio-iommu specification. The main
advantage of page tables over the MAP/UNMAP interface is sharing virtual
address spaces (SVA aka SVM), where an endpoint with PASID and I/O page
fault support can directly access process address spaces, and
application don't have to explicitly deal with DMA buffers. Using page
tables for regular DMA is also faster than MAP/UNMAP because it removes
the context switch on MAP (while UNMAP is replaced by INVALIDATE), but
requires pinning all guest memory which prevents the host from
over-committing memory. Some endpoints support I/O page faults (PCIe PRI)
which removes the need for pinning guest memory, but also has a runtime
cost.

The best compromise for managing DMA for any endpoint probably remains DMA
pools in the guest (for example Linux' page_pool) managed with MAP/UNMAP,
since it works with all systems and endpoints, provides ideal runtime
performance while allowing the host to overcommit most guest memory. For
assigned endpoints, support for page table isn't necessarily a net gain
outside of SVA but is certainly worth exploring. And for virtual
endpoints, page tables are more efficient than MAP/UNMAP.


Some reasons to add page tables to virtio-iommu:

* Although it can be done with emulated IOMMUs (vSMMU, VT-d, etc), several
  VMMs aim to avoid complex emulations and need this support in
  virtio-iommu (crosvm, cloud-hypervisor, kvmtool). Ideally those VMMs
  just transfer IOMMU commands between the virtqueue and the host API
  without having to deal with too much vendor detail. A VMM can also
  offload this to a vhost-iommu implementations.

* Ability to support a mix of hardware and software IOMMUs with differing
  configuration. The virtio-iommu driver discovers properties of the
  underlying IOMMU for each endpoint, for example the platform could have
  an IOMMU that supports nesting translation, an IOMMU that doesn't, and
  several virtual endpoints implemented by the VMM. One virtio-iommu
  instance can handle all of them and use the most advanced feature for
  each endpoint.

* Possibility of paravirtualized extensions (more efficient TLB
  management, optimized page requests). For virtual endpoints, even the
  basic page table support in virtio-iommu seemed more efficient than
  emulated IOMMUs in my tests.


This is the second version of the page table support for virtio-iommu, and
is quite different from v1 [1]. Mainly we added support for Intel, RISC-V
and AMD IOMMUs in addition to Arm. Although they don't need to be merged
at the same time, it's good to have an idea of the whole page table
support, since those architectures differ significantly.

The PROBE properties now contains raw ID registers (sanitized to only
contain information relevant to the guest). As a consequence SMMUv2 is not
supported anymore, would need to be added separately. ATTACH_TABLE is
also a little less generic, though I did try to describe the general
PASID table / page table model in the ATTACH_TABLE section rather than
in each format description. INVALIDATE is mostly unchanged.


You can find a PDF with these changes and diff colors here:

https://jpbrucker.net/virtio-iommu/spec/virtio-iommu-pt-v1.6-diff.pdf
https://jpbrucker.net/virtio-iommu/spec/virtio-iommu-pt-v1.6.pdf
https://jpbrucker.net/git/virtio/log/?h=viommu/tables

Please prototype and review!

It would also be good to involve some AMD folks, I didn't know who to Cc
and the AMD support is mostly illustrative for now. 

And do subscribe to the virtio-comment list [2] before replying,
otherwise your mail won't reach the list.

[1] https://lore.kernel.org/virtio-dev/20210205175812.143886-1-jean-philippe@linaro.org/
[2] https://www.oasis-open.org/committees/comments/index.php?wg_abbrev=virtio

Jean-Philippe Brucker (8):
  virtio-iommu: Add ATTACH_TABLE request
  virtio-iommu: Add INVALIDATE request
  virtio-iommu: Add Arm SMMUv3 support
  virtio-iommu: Add PASID field to DETACH
  virtio-iommu: Add RISC-V page tables
  virtio-iommu: Add AMD page tables
  virtio-iommu: Add VIRTIO_IOMMU_RESV_MEM_T_IDENTITY
  virtio-iommu: Add PASID information to fault report

Tina Zhang (1):
  virtio-iommu: Add VT-d IO page table

 device-types/iommu/description.tex   | 369 +++++++++++++++++++++++++--
 device-types/iommu/pgtable-amd.tex   | 187 ++++++++++++++
 device-types/iommu/pgtable-arm.tex   | 148 +++++++++++
 device-types/iommu/pgtable-intel.tex |  98 +++++++
 device-types/iommu/pgtable-riscv.tex | 111 ++++++++
 introduction.tex                     |   9 +
 6 files changed, 895 insertions(+), 27 deletions(-)
 create mode 100644 device-types/iommu/pgtable-amd.tex
 create mode 100644 device-types/iommu/pgtable-arm.tex
 create mode 100644 device-types/iommu/pgtable-intel.tex
 create mode 100644 device-types/iommu/pgtable-riscv.tex

-- 
2.42.0