Re: [PATCH v2 0/2] transport-pci: Introduce legacy registers access using AQ

[Parav Pandit <parav@nvidia.com>]

On 5/9/2023 11:51 PM, Jason Wang wrote:
> On Tue, May 9, 2023 at 11:56âAM Parav Pandit <parav@nvidia.com> wrote:
> > SIOV does not require mediation.
> > Composition is optional like VFs.
>
> This is not what I read from SIOV spec.
SIOV R1 spec was defined by _a_ vendor and it is undergoing upgrade to 
more practical use now. So we should let them finish the work.

It is hard to comment in this forum about it.

> > > > Hence, the transport we built is to consider this in mind for the
> > > > coming future.
> > >
> > > For transport virtqueue, it's not specific to PCI. It could be used in a much
> > > broader use case.
> > May be.
> > More below.
> >
> > > > So if each VF has its own configq, or cmdq, it totally make sense to
> > > > me which is bootstrap interface to transport existing config space interface.
> > > > The problem is: it is not backward compatible; Hence a device has no
> > > > way of when to support both or only new configq.
> > >
> > > Providing compatibility in software is much more simpler than inventing new
> > > hardware interfaces. Isn't it? (e.g if we want to provide compatibility for VF on
> > > a SIOV device). And inventing a new hardware interface for compatibility might
> > > not always work, it may break the advantages of the new hardware (like
> > > scalability).
> > What I proposed below is new hardware interface for new features.
> > Not for compatibility.
>
> I'm confused, the proposal is for legacy drives so it's for
> compatibility for sure. No?
This proposal is to get legacy devices to work over a PCI VF.

a configq/cmdq discussion is for new features for PF, VF, and SF/SIOV 
devices to work in _non_ backward compatible way, because for new 
features there is no such thing as backward compatibility between guest 
driver and the device.

> > Compatibility is coming at a cost which is not scaling.
> > If you attempt to scale, it breaks the future path forward to go without mediation.
> >
> > > > So eve growing these fields and optionally placement on configq
> > > > doesn't really help and device builder to build it efficiently
> > > > (without much predictability).
> > >
> > > Config queue is not the only choice, we have a lot of other choices (for example
> > > PASID may help to reduce the on-chip resources).
> > PASID is for process isolation security construct, it is not for transportation.
>
> I meant with PASID you don't even need a BAR.
Not sure I fully understand, but my guess is, this is coming from 
mediation thought process.

> > Same, SIOV and VFs do not prefer mediation going forward in the use cases we come across.
>
> Unless you don't want to provide VF compatibility for SIOV.
I do not understand what compatibility is this between which two elements?
VF has its identify currently and SIOV will have its own identity in future.

> > > Just to be sure we're on the same page. The proposal of both you and mine are
> > > based on the adminq for PF not VF. The reason is obvious:
> > > adminq per VF won't work without PASID, since it would have security issues.
> > The current proposal for legacy should be seen as separate and not to intermix with per VF based configuration queue.
> >
> > adminvq/config/control vq per VF and SIOV both devices will work without PASID (unrelated to legacy).
> > Because they are like any other queue, such as txq, rxq, cvq. All of these queues are non-mediated today for PF, VF devices.
> > (And so additional configq follows the natural model for config space access).
>
> This is only true if:
>
> 1) you don't want to provide any backward compatibility for current
> PCI transport, 
There is no concept of backward compatibility for new features.
There will be new driver anyway in the guest.

yes, I don't see a point in mediating 1.x config space for existing 
drivers as it is requires mediation.

> this means you need to use new drivers in the guest
> 2) you don't want to do live migration
Not sure how you assert this.

Live migration at VF level for the passthrough device can be just fine 
without any mediation as long as all the parameters on src and dst side 
match.
We already discussed in v0, so I prefer to avoid this again and focus on 
the patch in discussion.

> If you need one of the above, PASID is a must.

A PCI VF passthrough device LM without mediation is achievable without 
PASID for all the existing defined features using config space.

and also possible when new features are communicated over configq/cmdq.

Transporting existing config space using some additional queue is _not_ 
the objective, because building a scalable system where device has zero 
knowledge about needed backward compatibility is hard.
I provided the example of 100 Vfs with 30 and (30+40) bytes config 
layout previously.

Yet we are repeating and diverging from the discussion, that we should 
not intermix:
1. How to access legacy registers of the VF
vs
2. How to access existing or new configuration of the VF

Because both have very different requirements.