Re: [virtio-comment] Re: [PATCH v1 3/8] device-context: Define the device context fields for device migration

["Zhu, Lingshan" <lingshan.zhu@intel.com>]

On 10/20/2023 8:47 PM, Parav Pandit wrote:
> > From: Zhu, Lingshan <lingshan.zhu@intel.com>
> > Sent: Friday, October 20, 2023 4:42 PM
> >
> > On 10/20/2023 5:41 PM, Michael S. Tsirkin wrote:
> > > On Fri, Oct 20, 2023 at 05:31:01PM +0800, Zhu, Lingshan wrote:
> > > > On 10/19/2023 6:33 PM, Parav Pandit wrote:
> > > > > > From: Zhu, Lingshan <lingshan.zhu@intel.com>
> > > > > > Sent: Thursday, October 19, 2023 2:48 PM
> > > > > >
> > > > > > On 10/19/2023 5:14 PM, Michael S. Tsirkin wrote:
> > > > > > > On Thu, Oct 19, 2023 at 09:13:16AM +0000, Parav Pandit wrote:
> > > > > > > > > Oh, really? Quite interesting, do you want to move all config
> > > > > > > > > space fields in VF to admin vq? Have a plan?
> > > > > > > > Not in my plan for spec 1.4 time frame.
> > > > > > > > I do not want to divert the discussion, would like to focus on
> > > > > > > > device
> > > > > > migration phases.
> > > > > > > > Lets please discuss in some other dedicated thread.
> > > > > > > Possibly, if there's a way to send admin commands to vf itself
> > > > > > > then Lingshan will be happy?
> > > > > > still need to prove why admin commands are better than registers.
> > > > > Virtio spec development is not proof based approach. Please stop asking for
> > it.
> > > > > I tried my best to have technical answer in [1].
> > > > > I explained that registers simply do not work for passthrough mode
> > > > > (if this is what you are asking when you are asking prove its better).
> > > > > They can work for non_passthrough mediated mode.
> > > > >
> > > > > A member device may do admin commands using registers. Michael and I
> > are discussing presently in the same thread.
> > > > > Since there are multiple things to be done for device migration, dedicated
> > register set for each functionality do not scale well, hard to maintain and
> > extend.
> > > > > A register holding a command content make sense.
> > > > >
> > > > > Now, with that, if this can be useful only for non_passthrough, I made
> > humble request to transport them using AQ, this way, you get all benefits of AQ.
> > > > > And trying to understand, why AQ cannot possible or inferior?
> > > > >
> > > > > If you have commands like suspend/resume device, register or queue
> > transport simply donât work, because it's wrong to bifurcate the device with
> > such weird API.
> > > > > If you want to biferacate for mediation software, it probably makes sense to
> > operate at each VQ level, config space level. Such are very different commands
> > than passthrough.
> > > > > I think vdpa has demonstrated that very well on how to do specific work for
> > specific device type. So some of those work can be done using AQ.
> > > > > [1]
> > > > > https://lore.kernel.org/virtio-comment/870ace02-f99c-4582-932f-bd103
> > > > > 362dae9@intel.com/T/#m37743aa924536d0256d6b3b8e83a11c750f28794
> > > > We have been through your statement for many times.
> > > > This is not about how many times you repeated, if you think this is
> > > > true, you need to prove that with solid evidence.
> > > >
> > > >
> > > > For pass-through, I still recommend you to take a reference of
> > > > current virito-pci implementation, it works for pass-through, right?
> > > Current migration implementation in e.g. QEMU? It does but it traps
> > > data path accesses. That, I think we can agree, should not be the only
> > > option to migrate.
> > OK, I am glad we agree that config space work for pass-through, hope we don't
> > need to discuss this anymore.
> > > > For scale, I already told you for many times that they are per-device
> > > > facilities. How can a per-device facility not scale?
> > > > vDPA works fine on config space.
> > > >
> > > > So, if you still insist admin vq is better than config space like in
> > > > other thread you have concluded, you may imply that config space
> > > > interfaces should be re-factored to admin vq.
> > > There are good arguments that yes, virtio needs a transport for config
> > > space that is DMA based as opposed to memory mapped based.  This is
> > > one of the things all vendors seem to prefer in IDPF so virtio should have the
> > option.
> > Do you really want to refactor virtio-pci common config fields to PF's admin vq?
> > E.g, do you really want to move queue_enable in virtio-pci common config
> > fields to PF's admin vq?
> No. Please read the response carefully.
> I said 'For non-backward compatible SIOV device of the future, yes, virtio-pci common config (non init registers) should be moved to a vq, located on the member device directly.'
> Notice the 'member device directly'.
> Not the PF admin vq.
I think this is a question to Michael and he answered.

We are talking about PCI, not SIOV, for SIOV we need transport vq.

Here again, we are introducing basic facilities for live migration, and 
the implementation is transport-specific.
> > Config space is control path, DMA is data-path, let's better not mix them, we
> > never expect to use config space to transfer data.
> And that control path is only for the init time configuration as correctly listed in the virtio spec as,
>
> " Device configuration space should only be used for initialization-time parameters.".
don't you know new field reset_vq is introduced to virtio common cfg? 
This is not only for initialization, right?

and your citation is from Appendix B. Creating New Device Types, are we 
creating a new device type?
> > So we need DMA to transfer data, for example I take advantages of device DMA
> > to logging dirty pages, This also applies to in-flight descriptors.
> Can you please explain via virtqueue cannot be used for DMA bulk data transfer as listed in virtio spec.
>
> " The mechanism for bulk data transport on virtio devices is pretentiously called a virtqueue"
what is your point? vq can do DMA, so what?