Re: [PATCH v4 0/4] virtio: Clean up scatterlists and use the DMA API

[Rusty Russell <rusty@rustcorp.com.au>]

"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Wed, Sep 03, 2014 at 04:12:01PM +0930, Rusty Russell wrote:
>> Andy Lutomirski <luto@amacapital.net> writes:
>> > There really are virtio devices that are pieces of silicon and not
>> > figments of a hypervisor's imagination [1].
>> 
>> Hi Andy,
>> 
>>         As you're discovering, there's a reason no one has done the DMA
>> API before.
>> 
>> So the problem is that ppc64's IOMMU is a platform thing, not a bus
>> thing.  They really do carve out an exception for virtio devices,
>> because performance (LOTS of performance).  It remains to be seen if
>> other platforms have the same performance issues, but in absence of
>> other evidence, the answer is yes.
>> 
>> It's a hack.  But having specific virtual-only devices are an even
>> bigger hack.
>> 
>> Physical virtio devices have been talked about, but don't actually exist
>> in Real Life.  And someone a virtio PCI card is going to have serious
>> performance issues: mainly because they'll want the rings in the card's
>> MMIO region, not allocated by the driver.
>
> Why? What's wrong with rings in memory?

AFAICT, the card would have to access guest memory to read it, using
multiple DMA cycles.  That's going to be slow.

>>  Being broken on PPC is really
>> the least of their problems.
>> 
>> So, what do we do?  It'd be nice if Linux virtio Just Worked under Xen,
>> though Xen's IOMMU is outside the virtio spec.  Since virtio_pci can be
>> a module, obvious hacks like having xen_arch_setup initialize a dma_ops pointer
>> exposed by virtio_pci.c is out.
>
> Well virtio could probe for xen, it's not a lot of code.

We could, but I think this is going to be a more general problem in
future.  x86 is heading down the IOMMU path, and they're likely to
suffer similarly.

>> I think the best approach is to have a new feature bit (25 is free),
>> VIRTIO_F_USE_BUS_MAPPING which indicates that a device really wants to
>> use the mapping for the bus it is on.  A real device would set this,
>> or it won't work behind an IOMMU.  A Xen device would also set this.
>> 
>> Thoughts?
>> Rusty.
>
> OK and it should then be active even if guest does not ack
> the feature (so in fact, it would have to be a mandatory feature).
> That can work, but I still find this a bit inelegant: this is
> a property of the platform, not of the device.

True.  If a device needs it though, we're no worse of having a device
which doesn't work if the driver understand the feature than we were
before.

Cheers,
Rusty.