[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-comment] Live Migration of Virtio Virtual Function
* Jason Wang (jasowang@redhat.com) wrote: > > å 2021/8/19 äå10:58, Michael S. Tsirkin åé: > > On Thu, Aug 19, 2021 at 10:44:46AM +0800, Jason Wang wrote: > > > > The PF device will have an option to quiesce/freeze the VF device. > > > > > > Is such design a must? If no, why not simply introduce those functions in > > > the VF? > > Many IOMMUs only support protections at the function level. > > Thus we need ability to have one device (e.g. a PF) > > to control migration of another (e.g. a VF). > > > So as discussed previously, the only possible "advantage" is that the DMA is > isolated. > > > > This is because allowing VF to access hypervisor memory used for > > migration is not a good idea. > > For IOMMUs that support subfunctions, these "devices" could be > > subfunctions. > > > > The only alternative is to keep things in device memory which > > does not need an IOMMU. > > I guess we'd end up with something like a VQ in device memory which might > > be tricky from multiple points of view, but yes, this could be > > useful and people did ask for such a capability in the past. > > > I assume the spec already support this. We probably need some clarification > at the transport layer. But it's as simple as setting MMIO are as virtqueue > address? > > Except for the dirty bit tracking, we don't have bulk data that needs to be > transferred during migration. So a virtqueue is not must even in this case. > > > > > > > If yes, what's the reason for making virtio different (e.g VCPU live > > > migration is not designed like that)? > > I think the main difference is we need PF's help for memory > > tracking for pre-copy migration anyway. > > > Such kind of memory tracking is not a must. KVM uses software assisted > technologies (write protection) and it works very well. For virtio, > technology like shadow virtqueue has been used by DPDK and prototyped by > Eugenio. > > Even if we want to go with hardware technology, we have many alternatives > (as we've discussed in the past): > > 1) IOMMU dirty bit (E.g modern IOMMU have EA bit for logging external device > write) > 2) Write protection via IOMMU or device MMU > 3) Address space ID for isolating DMAs What's the state of those - last time I chatted to anyone about IOMMUs doing protection, things were at the 'in the future' stage. Dave > Using physical function is sub-optimal that all of the above since: > > 1) limited to a specific transport or implementation and it doesn't work for > device or transport without PF > 2) the virtio level function is not self contained, this makes any feature > that ties to PF impossible to be used in the nested layer > 3) more complicated than leveraging the existing facilities provided by the > platform or transport > > Consider (P)ASID will be ready very soon, workaround the platform limitation > via PF is not a good idea for me. Especially consider it's not a must and we > had already prototype the software assisted technology. > > > > Might as well integrate > > the rest of state in the same channel. > > > That's another question. I think for the function that is a must for doing > live migration, introducing them in the function itself is the most natural > way since we did all the other facilities there. This ease the function that > can be used in the nested layer. > > And using the channel in the PF is not coming for free. It requires > synchronization in the software or even QOS. > > Or we can just separate the dirty page tracking into PF (but need to define > them as basic facility for future extension). > > > > > > Another answer is that CPUs trivially switch between > > functions by switching the active page tables. For PCI DMA > > it is all much trickier sine the page tables can be separate > > from the device, and assumed to be mostly static. > > > I don't see much different, the page table is also separated from the CPU. > If the device supports state save and restore we can scheduling the multiple > VMs/VCPUs on the same device. > > > > So if you want to create something like the VMCS then > > again you either need some help from another device or > > put it in device memory. > > > For CPU virtualization, the states could be saved and restored via MSRs. For > virtio, accessing them via registers is also possible and much more simple. > > Thanks > > > > > > > > > This publicly archived list offers a means to provide input to the > OASIS Virtual I/O Device (VIRTIO) TC. > > In order to verify user consent to the Feedback License terms and > to minimize spam in the list archive, subscription is required > before posting. > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org > List help: virtio-comment-help@lists.oasis-open.org > List archive: https://lists.oasis-open.org/archives/virtio-comment/ > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists > Committee: https://www.oasis-open.org/committees/virtio/ > Join OASIS: https://www.oasis-open.org/join/ > -- Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]