Re: [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v18] virtio-net: support inner header hash

["Michael S. Tsirkin" <mst@redhat.com>]

On Thu, Jun 22, 2023 at 06:50:16PM +0000, Parav Pandit wrote:
> 
> 
> > From: virtio-comment@lists.oasis-open.org <virtio-comment@lists.oasis-
> > open.org> On Behalf Of Michael S. Tsirkin
> > Sent: Thursday, June 22, 2023 2:40 PM
> > To: Parav Pandit <parav@nvidia.com>
> > Cc: Heng Qi <hengqi@linux.alibaba.com>; virtio-comment@lists.oasis-open.org;
> > virtio-dev@lists.oasis-open.org; Jason Wang <jasowang@redhat.com>; Yuri
> > Benditovich <yuri.benditovich@daynix.com>; Xuan Zhuo
> > <xuanzhuo@linux.alibaba.com>; Cornelia Huck <cohuck@redhat.com>
> > Subject: [virtio-comment] Re: [virtio-dev] Re: [virtio-comment] Re: [PATCH v18]
> > virtio-net: support inner header hash
> > 
> > On Thu, Jun 22, 2023 at 06:17:54PM +0000, Parav Pandit wrote:
> > >
> > >
> > > > From: virtio-dev@lists.oasis-open.org
> > > > <virtio-dev@lists.oasis-open.org> On Behalf Of Michael S. Tsirkin
> > > > Sent: Thursday, June 22, 2023 2:12 PM
> > >
> > > [..]
> > >
> > > > > > > The proposal for 1.4 is literally very simple as below.
> > > > > > > 1. All existing fields of cfg space stays in cfg space 2. Any
> > > > > > > new capabilities to be queried, query using a vq (aq, cfgvq,
> > whatevervq).
> > > > > > > 3. Optionally existing fields can be queries over vq of #2
> > > > > > > Once this arrive, no need for new GET commands.
> > > > > > > Till that time, don't keep infinitely grow the cfg space.
> > > > > > > Any next addition to cfg space, should work on defining the cfgvq.
> > > > > >
> > > > > > Simple, but short sighted. I know you guys don't support your
> > > > > > hardware for 10-
> > > > > > 20 years but for software people do.
> > > > > > And so "All existing fields of cfg space stays in cfg space" is
> > > > > > a bad idea simply because this does not allow removing things
> > > > > > from config space not in 10 not in
> > > > > > 20 years not ever.
> > > > > >
> > > > > #1 is for backward compat for existing drivers.
> > > > >  You missed about #3. Existing cfg space fields can be queries
> > > > > using the cfgvq
> > > > too.
> > > >
> > > > Then #1 does not matter. We can give devices choice.
> > > >
> > > > > >
> > > > > > Instead we need to allow two ways to access config space.  Teach
> > > > > > drivers about both, actually mandate supporting both.  And then
> > > > > > devices will make their own cost/benefit decision about which
> > > > > > features they
> > > > want to support in MMIO.
> > > > >
> > > > > If both method is mandated, I don't see benefit at all of two methods.
> > > >
> > > > Mandated for driver.
> > > > Benefit is for devices, they will have the choice which drivers to
> > > > support. In 10-
> > > > 20 years all drivers support cfg command and then people can start
> > > > shipping devices without MMIO access to any registers.
> > > >
> > > Until that point devices are forced to burn memory, which is not needed at all
> > in < 3 years.
> > 
> > No they are not. They can make their own decision which fields to support in
> > MMIO space.
> Existing fields used by existing drivers must be in MMIO.
> Device does not have a choice.

They can if they like mask some less important features reducing the footprint.
This will be the cost/benefit analysis each vendor does separately.

Case in point, if inner hash support is in 1.3, and linux drivers don't
really use that at all for a year or two because it's only used by
alibaba in their stack, and by the time linux starts using
them it also supports the new commands, then practically
devices do not care and can just mask the feature bit in MMIO.


> > If they want to cut at 1.3 time, they can, but they also can cut it at 1.2 time, this
> > means some features will not be accessible through MMIO only through the
> > new commands.
> >
> New commands services new fields by newer driver.
> Newer driver can call new command to get new and old fields both.

yes.

> So 1.3 and 1.4 devices cannot optimize for older drivers.


I don't know what this means.

>  
> > > Once cfgvq is present for new fields, from the day 1, device does not need to
> > store any newly defined fields on MMIO.
> > 
> > Exactly.
> > 
> > > And 10 to 20 years, it can stop for existing fields..
> > >
> > > The clear benefit is fields defined in 1.4 no longer needs to be stored starting
> > from 2023-24.
> > 
> > And why is it a problem that someone can also build a 1.4 device with MMIO?
> >
> Why to build device using large MMIO when those fields are accessible via vq.

If you don't want to - don't. But it's simple and handy for software.
Not all devices are complex beasts like net.

Apropos, I am not yet sure the best way is simply adding admin vq to
vfs, in that this only works for fields not needed before DRIVER_OK.
Ideally I think we should fix all of config space, including device and
common config. I have some ideas but let's not get ahead of ourselves.

> VQ construct exists so it is already simple.
> 
> If the spec is defined in a way, that new fields must be accessible via vq, and optionally via MMIO, than it is ok.
> One can choose to build via MMIO.
> So MMIO for new fields must not be mandatory.

For devices, yes. I am suggesting mandating it for drivers.
Or maybe it's enough to make it a SHOULD.

-- 
MST