RE: [virtio-comment] Re: [PATCH 09/11] transport-pci: Describe PCI MMR dev config registers

[Parav Pandit <parav@nvidia.com>]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Thursday, April 13, 2023 1:20 PM
> 
> On Thu, Apr 13, 2023 at 01:14:15PM +0800, Jason Wang wrote:
> > > >>>> The proposed solution in this series enables it and avoid per
> > > >>>> field sw
> > > >>> interpretation and mediation in parsing values etc.
> 
> ... except for reset, notifications, and maybe more down the road.
> 
Your AQ proposal addresses reset too.
Nothing extra for the notifications, as comes for free from the device side.
> 
> > > >>> I don't think it's possible. See the discussion about
> > > >>> ORDER_PLATFORM and ACCESS_PLATFORM in previous threads.
> > > >>>
> > > >> I have read the previous thread.
> > > >> Hypervisor will be limiting to those platforms where ORDER_PLATFORM
> is not needed.
> > > >
> > > > So you introduce a bunch of new facilities that only work on some
> > > > specific archs. This breaks the architecture independence of
> > > > virtio since 1.0.
> > > The defined spec for PCI device does not work today for transitional
> > > device for virtualization. Only works in limited PF case.
> > > Hence this update.
> >
> > I fully understand the motivation. I just want to say
> >
> > 1) compare to the MMIO ar BAR0, this proposal doesn't provide much
> > advantages
> > 2) mediate on top of modern devices allows us to not worry about the
> > device design which is hard for legacy
> 
> I begin to think so, too. When I proposed this it looked like just a single
> capability will be enough, without a lot of mess.  But it seems that addressing
> this fully is getting more and more complex.
> The one thing we can't do in software is different header size for virtio net. For
> starters, let's add a capability to address that?

Hdr bit doesn't solve it because hypervisor is not involved in any trapping of feature bits, cvq or other vqs.
It is unified code for 1.x and transitional in hypervisor.

We have two options to satisfy the requirements.
(partly taken/repeated from Jason's yday email).

1. AQ (solves reset) + notification for building non transitional device that support perform well and it is both backward and forward compat
Pros:
a. efficient device reset.
b. efficient notifications from OS to device
c. device vendor doesn't need to build transitional configuration space.
d. works without any mediation in hv for 1.x and non 1.x for all non-legacy interfaces (vqs, config space, cvq, and future features).
e. can work with non-Linux guest VMs too

Cons:
a. More AQ commands work in sw
b. Does not work for bare metal PFs

2. Allowing MMIO BAR0 on transitional device as SHOULD requirement with larger BAR size.
Pros:
a. Can work with Linux bare-metal and Linux guest VMs as one of the wider uses case
b. in-efficient device handling for notifications
c. Works without mediation like 1.d.
d. Also works without HV mediation.

Cons:
a. device reset implementation is very for the hw.
b. requires transitional device to be built.
c. Notification performance may suffer.

For Marvell and us #1 works well.
I am evaluating #2 and get back.