Re: [virtio-dev] Re: [PATCH V2 2/2] virtio-pci: add PASID configuration extended capability

["Michael S. Tsirkin" <mst@redhat.com>]

On Fri, Jan 14, 2022 at 09:43:11AM +0000, Jean-Philippe Brucker wrote:
> Hi Jason,
> 
> On Thu, Jan 13, 2022 at 09:24:16AM +0800, Jason Wang wrote:
> > The device MUST use PASID TLP prefix for all memory transactions
> > initiated by the virtqueue that belong to a virtqueue group ...
> > 
> > > vring reads/writes and data buffer reads/writes come
> > > to mind. Virtqueue MSI-X messages are probably not included? Anything
> > > else?
> > 
> > According to the PCIe spec and the semantic, the PASID TLP prefix
> > should be used for
> > 
> > Memory Requests (including AtomicOp Requests) with Untranslated Addresses.
> > Address Translation Requests, ATS Invalidation Messages, Page Request
> > Messages, and PRG Response Messages
> > 
> > So we probably don't need to differentiate MSI-X messages with DMA
> > here. That's why I think a general "memory transaction" should be
> > sufficient here. If you don't agree, I can clarify it as what PCIe
> > spec did.
> 
> I think we should be explicit about this particular case because someone
> implementing this extension might get it wrong. MSIs should not have a
> PASID because IOMMUs don't support it:
> 
> * VT-d treats Requests-with-PASID to the special range 0xfeexxxxx as
>   normal DMA (3.14 Handling Requests to Interrupt Address Range)
> * AMD IOMMU reports an error for MSIs with PASID (2.2.7.7 PCIe® TLP PASID
>   Prefix).

Ouch. I didn't realize. Really weird, of the "what were they thinking"
variety. The natural thing would be to have PASID==VM and scope
both DMA and MSI within PASID. I guess that is precluded on these
platforms then.

> * Arm requires creating mappings to the MSI controller in the SMMU. This
>   has implications for SVA where the PASID accesses the whole process
>   address space: if MSI transactions have a PASID prefix, that requires
>   mapping the MSI controller into the process address space on Arm, which
>   isn't convenient.

I'd like to understand this part a bit better. Can you explain?
We are talking about the IOMMU address space right?
What is wrong with mapping the MSI controller there?
Isn't convenient in what sense?

> 
> Thanks,
> Jean