Re: [virtio-comment] Re: [PATCH v1 3/8] device-context: Define the device context fields for device migration

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Oct 18, 2023 at 10:22:57AM +0000, Parav Pandit wrote:
> 
> > From: Michael S. Tsirkin <mst@redhat.com>
> > Sent: Wednesday, October 18, 2023 3:26 PM
> 
> > For completeness, and to shorten the thread, can you please list known
> > issues/use cases that are addressed by the status bit interface and how you plan
> > for them to be addressed?
> 
> I will avoid listing known issues for a moment for status bit in this email.
> 
> Status bit interface helps in following good ways.
> 1. suspend/resume the device fully by the guest by negotiating the new feature.
> This can be useful in the guest-controlled PM flows of suspend/resume.
> I still think for this, only feature bit is necessary, and device_status modification is not needed.
> D0->D3 and D3->D0 transition of the pci can suspend and resume the device which can preserve the last device_status value before entering D3.
> (Like preserving all rest of the fields of common and other device config).
> This is orthogonal and needed regardless of device migration.
> 
> 2. If one does not want to passthrough a member device, but build a mediation-based device on top of existing virtio device,
> It can be useful with mediating software.
> Here the mediating software has ample duplicated knowledge of what the member device already has.
> This can fulfil the nested requirement differently provided a platform support it.
> (PASID limitation will be practical blocker here).
> 
> How to I plan to address above two?
> a. #1 to be addressed by having the _F_PM bit, when the bit is negotiated PCI PM drives the state.
> This will work orthogonal to VMM side migration and will co-exist with VMM based device migration.

OK that sounds kind of reasonable. Lingshan, Jason are you interested in
suspend/resume? Want to start a thread on best way to support that?

> b. nested use case:
> L0 VMM maps a VF to L1 guest as PF with emulated SR-IOV capability.
> L1 guest to enable SR-IOV and mapping the VF to L2 guest.
> Consulting industry ecosystem to support nested outside of virtio.

Can't say I like this much, *a lot* of things to implement,
and burning up a VF for control path is not nice.
As an alternative, I suggest a new admin command pci capability
with basically a PA and a valid bit. Easy to emulate and add to
a VF. And maybe some way to suggest a safe place for it that
won't conflict with anything? Still trying to figure out if
we should add PASID in there, or what. Maybe optionally?
If actual hardware does it we'd be burning up 20 bits,
but for a software implementation it's free.

-- 
MST