RE: [virtio-comment] Re: [PATCH v1 3/8] device-context: Define the device context fields for device migration

[Parav Pandit <parav@nvidia.com>]

> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> Sent: Friday, October 20, 2023 4:42 PM
> 
> On 10/20/2023 5:41 PM, Michael S. Tsirkin wrote:
> > On Fri, Oct 20, 2023 at 05:31:01PM +0800, Zhu, Lingshan wrote:
> >>
> >> On 10/19/2023 6:33 PM, Parav Pandit wrote:
> >>>> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> >>>> Sent: Thursday, October 19, 2023 2:48 PM
> >>>>
> >>>> On 10/19/2023 5:14 PM, Michael S. Tsirkin wrote:
> >>>>> On Thu, Oct 19, 2023 at 09:13:16AM +0000, Parav Pandit wrote:
> >>>>>>> Oh, really? Quite interesting, do you want to move all config
> >>>>>>> space fields in VF to admin vq? Have a plan?
> >>>>>> Not in my plan for spec 1.4 time frame.
> >>>>>> I do not want to divert the discussion, would like to focus on
> >>>>>> device
> >>>> migration phases.
> >>>>>> Lets please discuss in some other dedicated thread.
> >>>>> Possibly, if there's a way to send admin commands to vf itself
> >>>>> then Lingshan will be happy?
> >>>> still need to prove why admin commands are better than registers.
> >>> Virtio spec development is not proof based approach. Please stop asking for
> it.
> >>>
> >>> I tried my best to have technical answer in [1].
> >>> I explained that registers simply do not work for passthrough mode
> >>> (if this is what you are asking when you are asking prove its better).
> >>> They can work for non_passthrough mediated mode.
> >>>
> >>> A member device may do admin commands using registers. Michael and I
> are discussing presently in the same thread.
> >>>
> >>> Since there are multiple things to be done for device migration, dedicated
> register set for each functionality do not scale well, hard to maintain and
> extend.
> >>> A register holding a command content make sense.
> >>>
> >>> Now, with that, if this can be useful only for non_passthrough, I made
> humble request to transport them using AQ, this way, you get all benefits of AQ.
> >>> And trying to understand, why AQ cannot possible or inferior?
> >>>
> >>> If you have commands like suspend/resume device, register or queue
> transport simply donât work, because it's wrong to bifurcate the device with
> such weird API.
> >>> If you want to biferacate for mediation software, it probably makes sense to
> operate at each VQ level, config space level. Such are very different commands
> than passthrough.
> >>> I think vdpa has demonstrated that very well on how to do specific work for
> specific device type. So some of those work can be done using AQ.
> >>>
> >>> [1]
> >>> https://lore.kernel.org/virtio-comment/870ace02-f99c-4582-932f-bd103
> >>> 362dae9@intel.com/T/#m37743aa924536d0256d6b3b8e83a11c750f28794
> >> We have been through your statement for many times.
> >> This is not about how many times you repeated, if you think this is
> >> true, you need to prove that with solid evidence.
> >>
> >>
> >> For pass-through, I still recommend you to take a reference of
> >> current virito-pci implementation, it works for pass-through, right?
> > Current migration implementation in e.g. QEMU? It does but it traps
> > data path accesses. That, I think we can agree, should not be the only
> > option to migrate.
> OK, I am glad we agree that config space work for pass-through, hope we don't
> need to discuss this anymore.
> >
> >> For scale, I already told you for many times that they are per-device
> >> facilities. How can a per-device facility not scale?
> >> vDPA works fine on config space.
> >>
> >> So, if you still insist admin vq is better than config space like in
> >> other thread you have concluded, you may imply that config space
> >> interfaces should be re-factored to admin vq.
> > There are good arguments that yes, virtio needs a transport for config
> > space that is DMA based as opposed to memory mapped based.  This is
> > one of the things all vendors seem to prefer in IDPF so virtio should have the
> option.
> Do you really want to refactor virtio-pci common config fields to PF's admin vq?
> E.g, do you really want to move queue_enable in virtio-pci common config
> fields to PF's admin vq?
> 
No. Please read the response carefully.
I said 'For non-backward compatible SIOV device of the future, yes, virtio-pci common config (non init registers) should be moved to a vq, located on the member device directly.'
Notice the 'member device directly'.
Not the PF admin vq.

> Config space is control path, DMA is data-path, let's better not mix them, we
> never expect to use config space to transfer data.
> 
And that control path is only for the init time configuration as correctly listed in the virtio spec as,

" Device configuration space should only be used for initialization-time parameters.".

> So we need DMA to transfer data, for example I take advantages of device DMA
> to logging dirty pages, This also applies to in-flight descriptors.
> 
Can you please explain via virtqueue cannot be used for DMA bulk data transfer as listed in virtio spec.

" The mechanism for bulk data transport on virtio devices is pretentiously called a virtqueue"