Re: [virtio-dev] RE: [PATCH v3 6/6] vhost-user: support registering external host notifiers

[Paolo Bonzini <pbonzini@redhat.com>]

On 19/04/2018 14:43, Liang, Cunming wrote:
>> 2. Memory barriers. Right now after updating the avail idx, 
>> virtio does smp_wmb() and then the MMIO write. Normal hardware
>> drivers do wmb() which is an sfence. Can a PCI device read bypass
>> index write and see a stale index value?
>
> A compiler barrier is enough on strongly-ordered memory platform. As
> it doesn't re-order store, PCI device won't see a stale index value.
> But a weakly-ordered memory needs sfence.

That is complicated then.  We need to define a feature bit and (in the
Linux driver) propagate it to vring_create_virtqueue's weak_barrier
argument.  However:

- if we make it 1 when weak barriers are needed, the device also needs
to nack feature negotiation (not allow setting the FEATURES_OK) if the
bit is not set by the driver.  However, that is not enough.  Live
migration assumes that it is okay to migrate a virtual machine from a
source that doesn't support a feature to a destination that supports it.
 In this case, it would assume that it is okay to migrate from software
virtio to hardware virtio.  This is wrong because the destination would
use weak barriers

- if we make it 1 when strong barriers are enough, software virtio
devices needs to be updated to expose the bit.  This works, including
live migration, but updated drivers will now go slower when run against
an old device that doesn't know the feature bit.

Maybe bump the PCI revision, so that only the new revision has the bit?

Thanks,

Paolo