Re: [virtio-comment] Re: [PATCH v1 3/8] device-context: Define the device context fields for device migration

[Jason Wang <jasowang@redhat.com>]

On Wed, Oct 25, 2023 at 4:33âPM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Tue, Oct 24, 2023 at 06:27:04PM +0800, Zhu, Lingshan wrote:
> >
> >
> > On 10/23/2023 7:32 PM, Michael S. Tsirkin wrote:
> >
> >     On Mon, Oct 23, 2023 at 06:03:10PM +0800, Zhu, Lingshan wrote:
> >
> >         config space, MMIO, registers work for years, what is wrong with them?
> >
> >     Nothing as such. They don't seem to be appropriate for all use-case
> >     where people want to utilize virtio. I think a new transport
> >     will be needed to address these.
> >
> > New transport for new type of devices for sure, like transport vq for SIOV.
> >
> > I agree admin vq or admin cmds are useful in some use cases, that is
> > another story, should be case by case.
> >
> > For now, let's don't talk about all-use cases, just for current task, for live
> > migration.
> >
> > So IMHO, I still think we should use config space registers to control live
> > migration process.
> >
> >
>
> No because it forces integrating migration process with device driver.
> Which is ok for some use-cases but not all of them.  Find some other
> control plane for this.
>
>
> >                 Config space is control path, DMA is data-path, let's better not mix them,
> >                 we never expect to use config space to transfer data.
> >
> >                 So we need DMA to transfer data, for example I take advantages of device DMA
> >                 to logging dirty pages, This also applies to in-flight descriptors.
> >
> >             As long as you do, I personally see little benefit to retrieve parts of
> >             state with memory mapped accesses.
> >
> >         registers only control, and I personally believe a single register is much
> >         better
> >         than processing admin commands, more light-weight, more reliable, working
> >         for years.
> >
> >     Yea. It would be, if we could do everything through that register.
> >     But we can't really. Migration has too much data to pass around
> >     for that to be reasonable.
> >
> > data are not transferred by registers, they only control.
> >
> > We transfer data by DMA, the device writes DMA dirty pages information(bitmap)
> > to host isolated memory region.
> >
>
>
> If you do that then I don't see any reason not to use admin
> commands for that - either through a vq or a simpler
> interface.

I think we need to agree that admin commands are the only interface
for any future features before we can have an agreement here.

My understanding is that it is optional for the transport that
requires administrative commands like provisioning etc. It is not
necessarily the interface for new features.

>
>
> >
> >         Config space interfaces are fundamental for virtio-pci.
> >
> >
> >     They are in fact fundamental to virtio. Multiple transports to
> >     use config space are also fundamental.
> >
> > I agree. So I also agree to build admin vq live migration solution based on our
> > basic facilities, as Jason ever proposed.
>
>
> I'm not sure it's even a vq. I suggest a minimal interface to send
> admin commands. Could be used by migration, as transport, and more.
>

It's better if we can do that below the layer of admin commands. For
example, we don't stick device status with any specific interface. We
can keep doing things like this.

Thanks