RE: [virtio-comment] Re: [PATCH v13] virtio-net: support inner header hash

[Parav Pandit <parav@nvidia.com>]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Wednesday, April 26, 2023 12:12 AM
> 
> 
> No, this is ROM, not RAM.
> 
For VFs it is not ROM because one might configure VF with different feature bits.

> > CVQ already exists and provides the SET command. There is no reason to do
> GET in some other way.
> 
> Spoken looking at just hardware cost :)
Not really. The CVQ is already there, no extra overheads.

> The reason is that this is device specific. Maintainance overhead and system
> RAM cost for the code to support this should not be ignored.
Sure. As above its not a new and such query occurs only once or extremely rare.
> 
> > Device has single channel to provide a value and hence it doesn't need any
> internal synchronization between two paths.
> >
> > So, if we add a new feature bit as VIRTIO_F_CFG_SPACE_OVER_AQ it is an
> improvement.
> > But it still comes with a cost which device cannot mitigate.
> > The cost is,
> > 1. a driver may not negotiate such feature bit, and device ends up burning
> memory.
> > 1.b Device provisioning becomes a factor of what some guests may use/not
> use/already using on the live system.
> >
> > 2. Every device needs AQ even when the CVQ exists.
> >
> > Hence, better to avoid expanding current structure for any new fields,
> specially which has the SET equivalent.
> >
> > But if we want to with the path of VIRTIO_F_CFG_SPACE_OVER_AQ, it is fine.
> > More things can evolve for generic things like config space over AQ.
> 
> I am not sure what does VIRTIO_F_CFG_SPACE_OVER_AQ mean, and what are
> these costs.  
What I had in mind is all the rarely used and/or one time used config registers are queries over Q interface.
A device exposes a feature bit indicating that it offers it via a q interface instead of MMIO.
A net device already has a CVQ and its almost always there, so utilizing an existing object to query makes perfect sense.
It can be argued other way that, hey other devices cannot benefit for it because they missed the CVQ.
So may be a AQ can service for all the device types.

> What I had in mind is extending proposed vq transport to support
> sriov. I don't see why we can not have exactly 0 bytes of memory per VF.
> 
VFs other than legacy purposes do not undergo mediation via PF.
Legacy is the only exception; newer things is direct communication for a PCI device PF, SRIOV VFs and SIOV devices.

> And if we care about single bytes of PF memory (do we? there's only one PF per
> SRIOV device ...), what we should do is a variant of pci transport that
> aggressively saves memory.
Users already deploy 16 to 30 PFs coming from single physical card today in single system.
Building things uniformly for PF, VF, SIOV devices is coming for free.

It is not only this byte, but we also see that device needs to offer many capabilities more than boolean flag, so putting non latency sensitive item on the MMIO area hurts overall.