Re: [virtio-comment] [PATCH v1 1/8] admin: Add theory of operation for device migration

[Jason Wang <jasowang@redhat.com>]

On Tue, Oct 17, 2023 at 11:26âAM Parav Pandit <parav@nvidia.com> wrote:
>
>
>
> > From: Jason Wang <jasowang@redhat.com>
> > Sent: Tuesday, October 17, 2023 7:23 AM
> >
> > On Fri, Oct 13, 2023 at 2:36âPM Parav Pandit <parav@nvidia.com> wrote:
> > >
> > >
> > > > From: Jason Wang <jasowang@redhat.com>
> > > > Sent: Friday, October 13, 2023 6:47 AM
> > > >
> > > > On Thu, Oct 12, 2023 at 6:58âPM Parav Pandit <parav@nvidia.com> wrote:
> > > > >
> > > > >
> > > > > > From: Zhu, Lingshan <lingshan.zhu@intel.com>
> > > > > > Sent: Thursday, October 12, 2023 3:51 PM
> > > > > >
> > > > > > On 10/11/2023 7:43 PM, Parav Pandit wrote:
> > > > > > >> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> > > > > > >> Sent: Wednesday, October 11, 2023 3:55 PM
> > > > > > >>>>>>> I donât have any strong opinion to keep it or remove it
> > > > > > >>>>>>> as most stakeholders
> > > > > > >>>>>> has the clear view of requirements now.
> > > > > > >>>>>>> Let me know.
> > > > > > >>>>>> So some people use VFs with VFIO. Hence the module name.
> > > > > > >>>>>> This sentence by itself seems to have zero value for the
> > > > > > >>>>>> spec. Just
> > > > drop it.
> > > > > > >>>>> Ok. Will drop.
> > > > > > >>>> So why not build your admin vq live migration on our config
> > > > > > >>>> space solution, get out of the troubles, to make your life easier?
> > > > > > >>>>
> > > > > > >>> Your this question is completely unrelated to this reply or
> > > > > > >>> you misunderstood
> > > > > > >> what dropping commit log means.
> > > > > > >> if you can rebase admin vq LM on our basic facilities, I
> > > > > > >> think you dont need to talk about vfio in the first place, so
> > > > > > >> I ask you to re-consider
> > > > > > Jason's proposal.
> > > > > > > I donât really know why you are upset with the vfio term.
> > > > > > > It is the use case of the cloud operator and it is listed to
> > > > > > > indicate how proposal
> > > > > > fits in a such use case.
> > > > > > > If for some reason, you donât like vfio, fine. Ignore it and move on.
> > > > > > >
> > > > > > > I already answered that I will remove from the commit log,
> > > > > > > because the
> > > > > > requirements are well understood now by the committee.
> > > > > > >
> > > > > > > Your comment is again unrelated (repeated) to your past two
> > questions.
> > > > > > >
> > > > > > > I explained you the technical problem that admin command (not
> > > > > > > admin VQ)
> > > > > > of basic facilities cannot be done using config registers
> > > > > > without any mediation layer.
> > > > > > OK, I pop-ed Jason's proposal to make everything easier, and I
> > > > > > see it is
> > > > refused.
> > > > > Because it does not work for passthrough mode.
> > > >
> > > > How and why? What's wrong with just passing through the newly
> > > > introduced 2 or 3 registers to guests?
> > > >
> > > If passed to the guest who is not involved in the live migration flow, cannot
> > operate the device.
> > > VF = member device = controlled function PF = owner device =
> > > controlling function Device migration commands from the hypervisor are
> > > not forwarded inside the guest.
> >
> > I don't see why.
> >
> > For example, can you explain how a virtio reset in guests can survive with your
> > proposal but not a virtio suspend?
> >
> In this proposal virtio reset goes directly from guest to the device without intervention of hypervisor.

Why can suspend go directly from guest to device then?

> When such device reset is done, it does not reset the device context, nor it clears the dirty page records, because they are done by the controlling function.
>
> > I'm not convinced that the scalability is broken by just having 2 or 3 more
> > registers. We all know MSIX requires much more than this.
> >
> Saying there is some other high resource consumer method exists, so lets consume more in new interface we do now, is not good approach.

This is self-contradictory and double standard. You allow #MSI-X
vectors to grow but not config?

> MSI-X on its v2 is underway. Hopefully it will be finished this year, which is already cutting down O(N) resources.

Why couldn't such a method be applied to config registers and others?

>
> > You can't solve all the issues in one series. As stated many times,
> I am not solving all issues in one series. This series builds the infrastructure.

It's you that is raising the scalability issue, and what you've
ignored is that such an "issue" has existed for many years and various
hardware has been built on top of that.

Again, we should make sure the function is correct before we can talk
about others, otherwise it would be an endless discussion.

>
> > if you really
> > care about the scalability, the correct way is to behave like a real transport
> > through virtqueue instead of trying to duplicate the functionality of transport
> > virtqueue slowly.
> >
> This will be impossible because no device will transport driver notifications using a virtqueue.
> Therefore, virtqueue is not some generic transport that does everything - as simple as that. hence there is no transport virtqueue.

You won't get such a wrong conclusion if you read that proposal.

>
> And virtqueue for bulk data transfer exists so no need to invent yet another thing without a good reason.

I don't understand why this is related to transport virtqueue anyhow,
it's also a queue interface, no?

Thanks