Re: [virtio-comment] Re: [PATCH v3 6/8] admin: Add theory of operation for write recording commands

["Michael S. Tsirkin" <mst@redhat.com>]

On Wed, Nov 15, 2023 at 04:42:56PM +0800, Zhu, Lingshan wrote:
> 
> 
> On 11/15/2023 4:05 PM, Michael S. Tsirkin wrote:
> > On Wed, Nov 15, 2023 at 03:59:16PM +0800, Zhu, Lingshan wrote:
> > > 
> > > On 11/15/2023 3:51 PM, Michael S. Tsirkin wrote:
> > > > On Wed, Nov 15, 2023 at 12:05:59PM +0800, Zhu, Lingshan wrote:
> > > > > On 11/14/2023 4:27 PM, Michael S. Tsirkin wrote:
> > > > > > On Tue, Nov 14, 2023 at 03:34:32PM +0800, Zhu, Lingshan wrote:
> > > > > > > > > So I can't
> > > > > > > > > believe it has good performance overall. Logging via IOMMU or using
> > > > > > > > > shadow virtqueue doesn't need any extra PCI transactions at least.
> > > > > > > > On the other hand they have an extra CPU cost.  Personally if this is
> > > > > > > > coming from a hardware vendor, I am inclined to trust them wrt PCI
> > > > > > > > transactions.  But anyway, discussing this at a high level theoretically
> > > > > > > > is pointless - whoever bothers with actual prototyping for performance
> > > > > > > > testing wins, if no one does I'd expect a back of a napkin estimate
> > > > > > > > to be included.
> > > > > > > if so, Intel has released productions implementing these interfaces years
> > > > > > > ago,
> > > > > > > see live migration in 4.1. IFCVF vDPA Implementation,
> > > > > > > https://doc.dpdk.org/guides-21.11/vdpadevs/ifc.html
> > > > > > > and
> > > > > > That one is based on shadow queue, right? Which I think this shows
> > > > > > is worth supporting.
> > > > > Yes, it is shadow virtqueue, I assume this is already mostly done,
> > > > > do you see any gaps we need to address in our series that we should work on?
> > > > > 
> > > > > Thanks
> > > > There were a ton of comments posted on your series.
> > > Hope I didn't miss anything. I see your latest comments are about vq states,
> > > as replied before, I think we can record the states by two le16 and the
> > > in-flight
> > > descriptor tracking facility.
> > I don't know why you need the le16. in-flight tracking should be enough.
> > And given it needs DMA I would try really hard to actually use
> > admin commands for this.
> we need to record the on-device avail_idx and used_idx, or
> how can the destination side know the device internal values.

Again you never documented what state the device is in so I can't really
say for sure.  But generally whenever a buffer is used the internal
values are written out to memory.

> > 
> > > For this shadow virtqueue, do you think I should address this in my V4?
> > > Like saying: acknowledged control commands through the control virtqueue
> > > should be recorded, and we want to use shadow virtqueue to track dirty
> > > pages.
> > What you need to do is actually describe what do you expect the device
> > to do when it enters this suspend state. since you mention control
> > virtqueue then it seems that there needs to be a device type
> > specific text explaining the behaviour. If so this implies there
> > needs to be a list of device types that support suspend
> > until someone looks at each type and documents what it does.
> On a second thought, shadow vqs are hypervisor behaviors, maybe should not
> be
> described in this device spec.
> 
> Since SUSPEND is in device status, so for now I see every type of device
> implements
> device_status should support SUSPEND. This should be a general facility.