Re: [PATCH v3 6/8] admin: Add theory of operation for write recording commands

[Jason Wang <jasowang@redhat.com>]

On Mon, Oct 30, 2023 at 9:21âPM Parav Pandit <parav@nvidia.com> wrote:
>
> During a device migration flow (typically in a precopy phase of the
> live migration), a device may write to the guest memory. Some
> iommu/hypervisor may not be able to track these written pages.
> These pages to be migrated from source to destination hypervisor.
>
> A device which writes to these pages, provides the page address record
> of the to the owner device. The owner device starts write
> recording for the device and queries all the page addresses written by
> the device.
>
> Fixes: https://github.com/oasis-tcs/virtio-spec/issues/176
> Signed-off-by: Parav Pandit <parav@nvidia.com>
> Signed-off-by: Satananda Burla <sburla@marvell.com>
> ---
> changelog:
> v1->v2:
> - addressed comments from Michael
> - replaced iova with physical address
> ---
>  admin-cmds-device-migration.tex | 15 +++++++++++++++
>  1 file changed, 15 insertions(+)
>
> diff --git a/admin-cmds-device-migration.tex b/admin-cmds-device-migration.tex
> index ed911e4..2e32f2c 100644
> --- a/admin-cmds-device-migration.tex
> +++ b/admin-cmds-device-migration.tex
> @@ -95,6 +95,21 @@ \subsubsection{Device Migration}\label{sec:Basic Facilities of a Virtio Device /
>  The owner driver can discard any partially read or written device context when
>  any of the device migration flow should be aborted.
>
> +During the device migration flow, a passthrough device may write data to the
> +guest virtual machine's memory, a source hypervisor needs to keep track of these
> +written memory to migrate such memory to destination hypervisor.
> +Some systems may not be able to keep track of such memory write addresses at
> +hypervisor level. In such a scenario, a device records and reports these
> +written memory addresses to the owner device. The owner driver enables write
> +recording for one or more physical address ranges per device during device
> +migration flow. The owner driver periodically queries these written physical
> +address records from the device.

I wonder how PA works in this case. Device uses untranslated requests
so it can only see IOVA. We can't mandate ATS anyhow.

Thanks