Re: [virtio-comment] Re: [PATCH V2 6/6] virtio-pci: implement dirty page tracking

["Michael S. Tsirkin" <mst@redhat.com>]

On Mon, Nov 06, 2023 at 12:06:39PM +0800, Zhu, Lingshan wrote:
> > > Intel production work with similar bitmap
> > > based dirty page tracking solution for years.
> > and then VMs became bigger and PML was introduced.
> So you agree we should track dirty pages through the platform facilities?
> I am glad to hear that!

I just said that I thought there's no PML in platform facilities and that
might be a problem. Am I wrong?

> > 
> > > Otherwise the device should report PFN which is not very practical.
> > Why not?
> Really? the device report PFN?
> What can happen if the device keep writing a small piece of memory???

then you just report the PFN once. Should work like PML really -
IOW devices maintains a bit per page internally and reports
PFN when bit is set.

> > 
> > >      And the resolution is apparently 8 pages? You have just multiplied
> > >      the migration bandwidth by a factor of 8.
> > > 
> > > No, as described in the comments, the tacking granularity is controlled by \
> > > field{gra_power}, one bit represents a page with page_size = 2^(12 +
> > > gra_power). This can also be used to reduce the size of the bitmap.
> > .. at the cost of increasing migration bandwidth.
> The device is very likely to write a neighbor page,

how likely? and e.g. with slab randomization too? please collect some
data and show it.

> and this happens
> everywhere for example CPU read 64 bytes aligned data.

CPUs don't need to send their cache across a bandwidth constrained
shared network.

> 
> This is a tradeoff

tradeoff between which two options?

> > 
> > > "To prevent a read-modify-write procedure, if a memory page is dirty,
> > > optionally the device is permitted to set the entire byte, which encompasses the relevant bit, to 1."
> > > 
> > > This is optional and DMA is very likely to write a neighbor page, and the device transmit a whole byte anyway
> > > when a bit is dirty.
> > > 
> > > How about we use platform dirty page tracking facility then implement this in virtio, as Jason suggested?
> > > 
> > Without something like PML it likely won't scale either.
> So that would be platform issue which we don't need to take care of?
> Intel VT-d can do this for sure.

Intel VT-d supports PML from the IOMMU? I didn't realize. Could you help
me find it in the doc please? Which hardware supports this in the field?
What about other vendors?

-- 
MST