RE: [virtio-comment] Re: [PATCH v1 3/8] device-context: Define the device context fields for device migration

[Parav Pandit <parav@nvidia.com>]

> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> Sent: Wednesday, October 18, 2023 12:22 PM
> 
> On 10/18/2023 2:41 PM, Parav Pandit wrote:
> >
> >> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> >> Sent: Wednesday, October 18, 2023 12:06 PM
> >>
> >> On 10/18/2023 1:02 PM, Parav Pandit wrote:
> >>>> From: virtio-comment@lists.oasis-open.org
> >>>> <virtio-comment@lists.oasis- open.org> On Behalf Of Zhu, Lingshan
> >>>> Sent: Monday, October 16, 2023 3:18 PM
> >>>>
> >>>> On 10/13/2023 7:54 PM, Parav Pandit wrote:
> >>>>>> From: Zhu, Lingshan <lingshan.zhu@intel.com>
> >>>>>> Sent: Friday, October 13, 2023 3:14 PM
> >>>>>>>>>> How do you transfer the ownership?
> >>>>>>>>> An additional ownership deletgation by a new admin command.
> >>>>>>>> if you think this can work, do you want to cook a patch to
> >>>>>>>> implement this before you submitting this live migration series?
> >>>>>>> I answered this already above.
> >>>>>> talk is cheap, show me your patch
> >>>>> Huh. We presented the infrastructure that migrates, 30+ device
> >>>>> types,
> >>>> covering device context ideas from Oracle.
> >>>>> Covering P2P, supporting device_reset, FLR, dirty page tracking.
> >>>>>
> >>>>> Please have some respect for other members who covered more ground
> >>>>> than
> >>>> your series.
> >>>>> What more? Apply the same nested concept on the member device as
> >>>> Michael suggested, it is nested virtualization maintain exact same
> semantics.
> >>>>> So a VF is mapped as PF to the L1 guest.
> >>>>> L1 guest can enable SR-IOV on it, and map one VF to L2 guest.
> >>>>>
> >>>>> This nested work can be extended in future, once first level
> >>>>> nesting is
> >>>> covered.
> >>>>>> Answer all questions above, if you think a management VF can
> >>>>>> work, please show me your patch.
> >>>>> The idea evolves from technical debate then pointing fingers like
> >>>>> your
> >>>> comment.
> >>>>> I think a positive discussion with Michael and a pointer to the
> >>>>> paper from
> >>>> Jason gave a good direction of doing _right_ nesting that follows
> >>>> two
> >> principles.
> >>>>> a. efficiency property
> >>>>> b. equivalence property
> >>>>>
> >>>>> (c. resource control is natural already)
> >>>>>
> >>>>> Both apply at VMM and at VM level enabling recursive
> >>>>> virtualization, by
> >>>> having VF that can act as PF inside the guest.
> >>>>> [1] https://dl.acm.org/doi/pdf/10.1145/361011.361073
> >>>> Please just show me your patch resolving these opens, how about
> >>>> start from defining virito-fs device context and your management VF?
> >>> As answered, device context infrastructure is done, per device
> >>> specific device-
> >> context will be defined incrementally.
> >>> I will not be including virtio-fs in this series. It will be done
> >>> incrementally in
> >> future utilizing the infrastructure build in this series.
> >> Done? How do you conclude this? You just tell me what is the full set
> >> of virito-fs device context now and how to migrate them.
> >>
> >> You cant? you refuse or you don't? Do you expect the HW designer to
> >> figure out by themself?
> > I wont be able to tell now as I donât think it is necessary for this series.
> > If one out of 30 devices cannot migrate because of unimaginable amount of
> complexity has been placed there, may be one will not implement it as member
> device.
> >
> >  From experience of migratable complex gpu devices, rdma devices (stateful
> having hundred thousand of stateful QPs), my understanding is complex state of
> virtio-fs can be defined and migratable.
> > Mlx5 driver consist of 150,000 lines of code and that device is migratable
> with complex state.
> > So I am optimistic that virtio-fs can be migratable too.
> > It does not have to limited by my limited creativity of 2023.
> > May be I am wrong, in that case one will not implement passthrough virtio-fs
> device.
> your series wants to migrate device context, but doesn't define device context,
> does this sounds reasonable?

Device generic context is defined at [1] and also the infrastructure for defining the device context in parallel by multiple people can be done post the work of [1].

Per each device type context will be defined incrementally post this work.

[1] https://lists.oasis-open.org/archives/virtio-comment/202310/msg00190.html