Re: [virtio-dev] RFC: Doorbell suppression, packed-ring mode and hardware offload

[David Riddoch <driddoch@solarflare.com>]

> > > > I would prefer to have the write barrier before writing the idx.
> > > Well that's driver overhead for something device might never utilise in
> > > a given workload. If we are optimizing let's optimize for speed.
> > I think doing the barrier before writing idx is best for speed (see below).
> I don't see it below :(

Sorry, I'm not being clear. If the write barrier is before the idx, 
then a PV device can read the idx, do a single rmb and read a whole 
bunch of descriptors. As things stand today a PV device has to do an 
rmb for each descriptor that it reads.

I'm not sure, but this may be what Jason meant when he said "prefetch".

> > But it is really desirable for complexity: Cases like this are easy to
> > handle in software, but much much harder in pipelined hardware
> > implementations.
> Maybe we should use a flag in event suppression structure.
> This way device can switch between being notification driven
> and being driven by index reads.

On first thought it seems hard to avoid races: On receiving a doorbell 
the device wishes to transition from doorbells to polling, so 
driver->avail_idx currently has an old value; device writes 
device->flags; a little later device reads driver->avail_idx. It might 
have a new value written by the driver, or an old value...the device 
can't tell. Needs more thought.

> > > I am also wondering: what would the analog of this feature be for split
> > > rings? We are burning a feature bit, might as well find a good
> > > use for it.
> > I don't have any suggestions. I worry that associating the same bit with a
> > split-ring feature would create an undesirable coupling: A device offering X
> > for packed-ring would also necessarily have to implement Y for split-ring
> > and vice versa (if both ring types are supported).
> BTW we really need to look more at how can devices support subsets
> of features. Right now devices can fail FEATURES_OK but drivers do
> not recover. Some dependencies are defined by spec and drivers
> can force them but devices can't.

Completely agree. In order to limit complexity in hardware 
implementations it would be nice to be able to express "I support X, and 
X+Y, but not Y by itself" and similar. There is no way to do that today.


--
David Riddoch  <driddoch@solarflare.com> -- Chief Architect, Solarflare