RE: [virtio-comment] RE: [PATCH V2 3/6] virtio: dont reset vqs when SUSPEND

[Parav Pandit <parav@nvidia.com>]

> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> Sent: Wednesday, November 22, 2023 3:34 PM
> 
> On 11/22/2023 2:47 PM, Parav Pandit wrote:
> >> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> >> Sent: Wednesday, November 22, 2023 7:22 AM
> >>
> >> On 11/22/2023 5:18 AM, Michael S. Tsirkin wrote:
> >>> On Tue, Nov 21, 2023 at 03:33:07PM +0800, Jason Wang wrote:
> >>>>> Lingshan claimed that suspending device is for live migration in
> >>>>> commit log
> >> and in discussion he portray it as some basic facility unrelated to
> >> device migration such as debug etc.
> >>>>> Instead of claiming it as some non_device_migration facility does
> >>>>> not make
> >> sense.
> >>>> It is used for migration for sure.
> >>> Well having a generic facility to stop device sounds like a nice thing.
> >>> However the devil is in the detail. A lot of detail here seems very
> >>> much tailored to a very specific implementation in mind.
> >>> So thinking through how it will work e.g. for power management would
> >>> be a good excercise to figure out how it should work in detail.
> >>> Parav did you indicate at some point a virtio specific SUSPEND bit
> >>> can be useful for PM? Could you share how it's better than transport
> >>> level PM and what the requirements are?
> >>> Thanks!
> >> Do you mean letting the device enter a new power state when SUSPEND,
> >> and such description in transport-pci.tex? Then resume normal state
> >> on DRIVER_OK.
> > My proposal is,
> > 1. suspend bit (not state) to be controlled by the guest driver
> The guest can set SUSPEND bit for sure. However there is no reason to forbid
> host from setting SUSPEND.
Sure. hypervisor can also touch all the bits it has access to.

> 
> The reason is, the live migration process should be transparent to the guest,
> that means the guest should be suspended first, then the host suspend the
> device, so no I/O errors in the guest.
I explained this yday to my best, that hypervisor in my proposal also must touch the device when the guest is NOT suspended.
This is helping to lower the downtime as base tenet.

(you can imagine as the pre-copy phase of the device context like how there is pre-copy is present for the memory).

so guest vcpus to suspend first before device is smaller case of what my proposal covers.
And it is fine.

I am ok with guest controlled suspend bit, that optionally may be used by the hypervisor.

> > 2. this bit must be busy poll type. Meaning,
> This is already in the patch, the driver should re-read to make sure the
> SUSPEND bit is set.
> > a. the driver must get acknowledgement from the device that the suspend
> operation is completed in the device.
> > b. the driver must get acknowledgement from the device that the resume
> operation is completed in the device.
> Yes, by re-read, for example the device should only present SUSPEND bit in the
> device status when it finished the process to suspend, already in the patch
> series.
> >
> > 3. Not to confuse this with administrative mode active/stop/freeze set
> > by the owner device during device migration
> >
> > 4. This feature is usable in power management use case and may be some
> other.
> Yes, as long as we use virtio status to control PM, rather than the reverse.

PCI PM is already there. The extra optional suspend bit is an aid to those virtio device which cannot meet the deadlines in 10msec timing.