RE: [virtio-comment] [PATCH v1 1/8] admin: Add theory of operation for device migration

[Parav Pandit <parav@nvidia.com>]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Thursday, October 19, 2023 2:41 PM
> 
> On Thu, Oct 19, 2023 at 08:58:10AM +0000, Parav Pandit wrote:
> > It cannot be in the PCI 4K config space for sure.
> > It must reside in the virtio config space.
> 
> Why? 
Because pci spec has clearly called out to not place any device specific things in there.

Citation in pci spec
" It is strongly recommended that PCI Express devices place no registers in Configuration Space other than those in
headers or Capability structures architected by applicable PCI specifications."

> My concern with virtio config space would be that it's not orthogonal to
> other things in the config space. E.g. you need to look at feature bits to discover
> presence. How does this work while you are documenting that device can
> undergo reset at any time?
This is why such solution cannot work for passthrough. It can only fulfil #2 based approach.

> With a capability you can discover it without poking at features.
> 
It is discouraged by pci spec.
Pci caps for mostly doing very small init time sort of config.
Not to run frequent commands.

> > I am sure that this is used for passthrough mode of #1.
> > So, can you please confirm to write this up for mode #2 only?
> 
> To me it sounds like a generally useful capability that could be used as basis e.g.
> for admin command transport.
> 
Unfortunately, it cannot be pci capability. 
It needs to stay in virtio area and only fulfill use case of #2.

> > > > A variation of that for the member device, there is owner device,
> > > > hence
> > > admin command on the AQ can be used.
> > > >
> > > > If we can converge on common virtio interface between #1 and #2, great.
> > > > If we cannot be due to technical issues, we shouldn't step on each
> > > > other's
> > > toes, instead build the two interfaces for two different use cases
> > > overcoming its own technical challenges.
> > > >
> > > > And when in future, someone want to implement different kind of
> > > > bisections,
> > > they can propose the extensions.
> > >
> > > Not good at all, this means the interface is very narrow.
> > > Your "propose an extension" just doesn't work practically.
> > > It takes years for things to be widely deployed in the field, by the
> > > time they are there are more use-cases.
> >
> > We usually see it getting deployed in < 1 year time with new spec
> advancement pace for many features.
> > Building something for unreasonable amount of time without use case results
> in missing the immediate deployments that happens in 2024 to 2027 of 1.4 spec
> time frame.
> >
> > > We need something universal and admin commands were supposed to be
> > > just this.
> > I don't see a universal solution for all problems for above #1 and #2.
> >
> > Solving above #2 will cover large part of deployments that users are doing.
> 
> OK. But additionally, if an interface can cover a couple of use-cases we can be
> reasonably sure it's going to cover more going forward.
May be.