RE: [PATCH v3 6/8] admin: Add theory of operation for write recording commands

[Parav Pandit <parav@nvidia.com>]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Thursday, November 16, 2023 10:56 PM
> 
> On Thu, Nov 16, 2023 at 04:26:53PM +0000, Parav Pandit wrote:
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Thursday, November 16, 2023 5:18 PM
> > >
> > > On Thu, Nov 16, 2023 at 07:40:57AM +0000, Parav Pandit wrote:
> > > >
> > > > > From: Michael S. Tsirkin <mst@redhat.com>
> > > > > Sent: Thursday, November 16, 2023 1:06 PM
> > > > >
> > > > > On Thu, Nov 16, 2023 at 12:51:40AM -0500, Michael S. Tsirkin wrote:
> > > > > > On Thu, Nov 16, 2023 at 05:29:54AM +0000, Parav Pandit wrote:
> > > > > > > We should expose a limit of the device in the proposed
> > > > > WRITE_RECORD_CAP_QUERY command, that how much range it can
> track.
> > > > > > > So that future provisioning framework can use it.
> > > > > > >
> > > > > > > I will cover this in v5 early next week.
> > > > > >
> > > > > > I do worry about how this can even work though. If you want a
> > > > > > generic device you do not get to dictate how much memory VM has.
> > > > > >
> > > > > > Aren't we talking bit per page? With 1TByte of memory to track
> > > > > > -> 256Gbit -> 32Gbit -> 8Gbyte per VF?
> > > > >
> > > > > Ugh. Actually of course:
> > > > > With 1TByte of memory to track -> 256Mbit -> 32Mbit -> 8Mbyte
> > > > > per VF
> > > > >
> > > > > 8Gbyte per *PF* with 1K VFs.
> > > > >
> > > > Device may not maintain as a bitmap.
> > >
> > > However you maintain it, there's 256Mega bit of information.
> > There may be other data structures that device may deploy as for example
> hash or tree or something else.
> 
> Point being?
The device may have some hashing accelerator or other improvements that may perform better than bitmap as many queues in parallel attempt to update the shared database.

> 
> > And this is runtime memory only during the short live migration period of
> 400msec or less.
> > It is not some _always_ resident memory.
> 
> No - write tracking is used in the live phase of migration. It can be enabled as
> long as you wish - it's a question of policy.  There actually exist solutions that
> utilize this phase for redundancy, permanently running in this mode.

If such use case exists, one may further improve the device implementation.