Re: [virtio-dev] Re: [PATCH 0/5] virtio: introduce SUSPEND bit and vq state

["Zhu, Lingshan" <lingshan.zhu@intel.com>]

On 9/22/2023 11:08 AM, Jason Wang wrote:
> On Thu, Sep 21, 2023 at 12:19âPM Parav Pandit <parav@nvidia.com> wrote:
> > > From: Jason Wang <jasowang@redhat.com>
> > > Sent: Thursday, September 21, 2023 9:39 AM
> > >
> > > On Thu, Sep 21, 2023 at 12:01âPM Parav Pandit <parav@nvidia.com> wrote:
> > > > > From: Jason Wang <jasowang@redhat.com>
> > > > > Sent: Thursday, September 21, 2023 8:48 AM
> > > > > As replied in another thread, the issues for BAR are:
> > > > >
> > > > > 1) Not sure it can have an efficient interface, it would be
> > > > > something like VIRTIO_PCI_CAP_PCI_CFG which is very slow compared to
> > > > > single register accessing
> > > > > 2) There's no owner/group/member for MMIO, most of the time, we only
> > > > > need a single MMIO device. If we want the owner to manage itself, it
> > > > > seems redundant as is implied in all the existing transports (without admin
> > > commands).
> > > > > Even if we had, it might still suffer from bootstrap issues.
> > > > > 3) For live migration, it means the admin commands needs to start
> > > > > from duplicating every existing transport specific interface it can
> > > > > give us. One example is that we may end up with two interfaces to
> > > > > access virtqueue addresses etc. This results in extra complicity and
> > > > > it is actually a full transport (driver can just use admin commands to drive
> > > the device).
> > > > In [1] there is no duplication. The live migration driver never parses the device
> > > context either while reading or write.
> > > > Hence no code and no complexity in driver and no duplicate work.
> > > > Therefore, those admin commands are not to drive the guest device either.
> > > I'm not sure how this is related to the duplication issue.
> > You commented that admin virtqueue duplicates somethings.
> > And I explained above that it does not.
> >
> > > > > 4) Admin commands itself may not be capable of doing things like
> > > > > dirty page logging, it requires the assistance from the transport
> > > > Admin command in [1] is capable of dirty page logging.
> > > In your design, the logging is done via DMA not the virtqueue.
> > No. it is done via admin command, not DMA in [2].
> >
> > [2] 20230909142911.524407-7-parav@nvidia.com/T/#m17b09acd8c73d374e98ad84764b315afa94f59c9">https://lore.kernel.org/virtio-comment/20230909142911.524407-7-parav@nvidia.com/T/#m17b09acd8c73d374e98ad84764b315afa94f59c9
> >
> > > The only job for virtqueue is to initiate the DMA. But if DMA can be initiated via
> > > virtqueue, it can be done in other ways.
> > Lets first establish 4 things in alternative way, any motivation to do so with 5th point without giant registers need in device.
> >
> > > > > 1) Parav's proposal does several couplings: couple basic build
> > > > > blocks (suspend, dirty page tracking) with live migration, couple
> > > > > live migration with admin commands.
> > > > In which use case you find dirty page tracking useful without migration for
> > > which you like to see it detached from device migration flow?
> > >
> > > Is it only the dirty page tracking? It's the combinations of
> > >
> > > 1) suspending
> > > 2) device states
> > > 3) dirty page tracking
> > >
> > > Eeah of those will have use cases other than live migration: VM stop, power
> > > management in VM, profiling and monitoring, failover etc.
> > Suspend/resume with different power state is driven by the guest directly.
> And there's hibernation actually where device states might be useful.
>
> > So it may find some overlap.
> >
> > Device context has no overlap.
> I can give you one example, e.g debugging.
>
> > Dirty page tracking has no overlap. What do you want to profile and monitor? In case if you want to profile, it can be used without migration command anyway?
> It works like a dirty bit of PTE. We all know it has a broader use
> case than logging. For example, tracking working set and do
> optimization on IOMMU/IOTLB or even device IOTLB.
>
> 1) Try to prove your facility can only work for one specific cases
> 2) Try to prove your facility can work for more than one cases
>
> Which one is easier and more beneficial to virtio?
>
>
> > If you describe, may be we I can split "device migration" chapter to two pieces,
> > Device management and device migration.
> >
> > Device migration will use these basic facility.
> > Would that help you?
> Definitely, but it needs to be done by not making it under the
> subsection of admin commands, that's it.
>
> Let me repeat once again here for the possible steps to collaboration:
>
> 1) define virtqueue state, inflight descriptors in the section of
> basic facility but not under the admin commands
> 2) define the dirty page tracking, device context/states in the
> section of basic facility but not under the admin commands
> 3) define transport specific interfaces or admin commands to access them
I totally agree with this proposal.
> Does this work? It seems you refused such steps in the past.
>
> Actually, I would like to leave 2) as it's very complicated which
> might not converge easily.
>
> Thanks