RE: [virtio-comment] Re: [PATCH 00/11] Introduce transitional mmr pci device

[Parav Pandit <parav@nvidia.com>]

> From: virtio-comment@lists.oasis-open.org <virtio-comment@lists.oasis-
> open.org> On Behalf Of Michael S. Tsirkin

> > Transport vq for legacy MMR purpose seems fine with its latency and DMA
> overheads.
> > Your question was about "scalability".
> > After your latest response, I am unclear what "scalability" means.
> > Do you mean saving the register space in the PCI device?
> 
> yes that's how you used scalability in the past.
>
Ok. I am aligned.
 
> > If yes, than, no for legacy guests for scalability it is not required, because the
> legacy register is subset of 1.x.
> 
> Weird.  what does guest being legacy have to do with a wish to save registers
> on the host hardware? 
Because legacy has subset of the registers of 1.x. So no new registers additional expected on legacy side.

> You don't have so many legacy guests as modern
> guests? Why?
> 
This isn't true.

There is a trade-off, upto certain N, MMR based register access is fine.
This is because 1.x is exposing super set of registers of legacy.
Beyond a certain point device will have difficulty in doing MMR for legacy and 1.x.
At that point, legacy over tvq can be better scale but with lot higher latency order of magnitude higher compare to MMR.
If tvq being the only transport for these registers access, it would hurt at lower scale too, due the primary nature of non_register access.
And scale is relative from device to device.

> >
> > > > > And presumably it can all be done in firmware ...
> > > > > Is there actual hardware that can't implement transport vq but
> > > > > is going to implement the mmr spec?
> > > > >
> > > > Nvidia and Marvell DPUs implement MMR spec.
> > >
> > > Hmm implement it in what sense exactly?
> > >
> > Do not follow the question.
> > The proposed series will be implemented as PCI SR-IOV devices using MMR
> spec.
> >
> > > > Transport VQ has very high latency and DMA overheads for 2 to 4
> > > > bytes
> > > read/write.
> > >
> > > How many of these 2 byte accesses trigger from a typical guest?
> > >
> > Mostly during the VM boot time. 20 to 40 registers read write access.
> 
> That is not a lot! How long does a DMA operation take then?
> 
> > > > And before discussing "why not that approach", lets finish
> > > > reviewing "this
> > > approach" first.
> > >
> > > That's a weird way to put it. We don't want so many ways to do
> > > legacy if we can help it.
> > Sure, so lets finish the review of current proposal details.
> > At the moment
> > a. I don't see any visible gain of transport VQ other than device reset part I
> explained.
> 
> For example, we do not need a new range of device IDs and existing drivers can
> bind on the host.
>
So, unlikely due to already discussed limitation of feature negotiation.
Existing transitional driver would also look for an IOBAR being second limitation.

> > b. it can be a way with high latency, DMA overheads on the virtqueue for
> read/writes for small access.
> 
> numbers?
It depends on the implementation, but at minimum, writes and reads can pay order of magnitude higher in 10 msec range.