Re: [virtio-comment] [PATCH 1/5] virtio: introduce vq state as basic facility

["Zhu, Lingshan" <lingshan.zhu@intel.com>]

On 9/6/2023 4:28 PM, Michael S. Tsirkin wrote:
> On Wed, Sep 06, 2023 at 04:16:33PM +0800, Zhu Lingshan wrote:
> > This patch adds new device facility to save and restore virtqueue
> > state. The virtqueue state is split into two parts:
> >
> > - The available state: The state that is used for read the next
> >    available buffer.
> > - The used state: The state that is used for make buffer used.
> >
> > This will simply the transport specific method implementation. E.g two
> > le16 could be used instead of a single le32). For split virtqueue, we
> > only need the available state since the used state is implemented in
> > the virtqueue itself (the used index). For packed virtqueue, we need
> > both the available state and the used state.
> >
> > Those states are required to implement live migration support for
> > virtio device.
> >
> > Signed-off-by: Zhu Lingshan <lingshan.zhu@intel.com>
> > Signed-off-by: Jason Wang <jasowang@redhat.com>
> > Signed-off-by: Eugenio PÃrez <eperezma@redhat.com>
> > ---
> >   content.tex | 65 +++++++++++++++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 65 insertions(+)
> >
> > diff --git a/content.tex b/content.tex
> > index 0a62dce..0e492cd 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -502,6 +502,71 @@ \section{Exporting Objects}\label{sec:Basic Facilities of a Virtio Device / Expo
> >   types. It is RECOMMENDED that devices generate version 4
> >   UUIDs as specified by \hyperref[intro:rfc4122]{[RFC4122]}.
> >   
> > +\section{Virtqueue State}\label{sec:Virtqueues / Virtqueue State}
> > +
> > +When VIRTIO_F_QUEUE_STATE has been negotiated, the driver can set and
> > +get the device internal virtqueue state through the following
> > +fields. The implementation of the interfaces is transport specific.
>
> virtqueue state can not, generally, be described by two 16 bit
> indices.
>
> Consider an example: these buffers available: A B C D
> After device used descriptors A and C, what is its state and
> how do you describe it using a single index?
<discussed in the 0/5 cover letter>
> > +
> > +\subsection{\field{Available State} Field}
> > +
> > +The available state field is two bytes of virtqueue state that is used by
> > +the device to read the next available buffer.
> > +
> > +When VIRTIO_RING_F_PACKED is not negotiated, it contains:
> > +
> > +\begin{lstlisting}
> > +le16 last_avail_idx;
> > +\end{lstlisting}
> > +
> > +The \field{last_avail_idx} field is the free-running available ring
> > +index where the device will read the next available head of a
> > +descriptor chain.
> next after what?
I am not sure I get it.
It's like a pointer, so I assume it goes without saying, implies the 
next "address".

Do you suggest "next after current being processed ones" or others?
> > +
> > +See also \ref{sec:Basic Facilities of a Virtio Device / Virtqueues / The Virtqueue Available Ring}.
> > +
> > +When VIRTIO_RING_F_PACKED is negotiated, it contains:
> > +
> > +\begin{lstlisting}
> > +le16 {
> > +  last_avail_idx : 15;
> > +  last_avail_wrap_counter : 1;
> > +};
> > +\end{lstlisting}
> > +
> > +The \field{last_avail_idx} field is the free-running location
> > +where the device read the next descriptor from the virtqueue descriptor ring.
> > +
> > +The \field{last_avail_wrap_counter} field is the last driver ring wrap
> > +counter that was observed by the device.
> > +
> > +See also \ref{sec:Packed Virtqueues / Driver and Device Ring Wrap Counters}.
> > +
> > +\subsection{\field{Used State} Field}
> > +
> > +The used state field is two bytes of virtqueue state that is used by
> > +the device when marking a buffer used.
> > +
> > +When VIRTIO_RING_F_PACKED is negotiated, the used state contains:
> > +
> > +\begin{lstlisting}
> > +le16 {
> > +  used_idx : 15;
> > +  used_wrap_counter : 1;
> > +};
> > +\end{lstlisting}
> > +
> > +The \field{used_idx} field is the free-running location where the device write the next
> > +used descriptor to the descriptor ring.
> I don't get what good does this used_idx do - used descriptors are written in
> order so just check which ones are valid?
a bit confused. please correct me if I misunderstood.

Valid to the device? avail_idx? How to speculate used_idx from avail_idx,
or walk through the ring?
> And driver does of course know what the used_wrap_counter is
> otherwise it can't work.
> Or is this for some kind of
> split driver setup where looking at the ring is impossible?
For splitted virtqueue, we don't need to migrate used_idx, but for
packed vq, is it easier if we record and restore it?
> > +
> > +The \field{used_wrap_counter} field is the wrap counter that is used
> > +by the device.
> > +
> > +See also \ref{sec:Packed Virtqueues / Driver and Device Ring Wrap Counters}.
> > +
> > +When VIRTIO_RING_F_PACKED is not negotiated, the 16-bit value of \field{used_idx}
> > +is always 0
> > +
> >   \input{admin.tex}
> >   
> >   \chapter{General Initialization And Device Operation}\label{sec:General Initialization And Device Operation}
> > --
> > 2.35.3
> >
> >
> > This publicly archived list offers a means to provide input to the
> > OASIS Virtual I/O Device (VIRTIO) TC.
> >
> > In order to verify user consent to the Feedback License terms and
> > to minimize spam in the list archive, subscription is required
> > before posting.
> >
> > Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> > List help: virtio-comment-help@lists.oasis-open.org
> > List archive: https://lists.oasis-open.org/archives/virtio-comment/
> > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> > Committee: https://www.oasis-open.org/committees/virtio/
> > Join OASIS: https://www.oasis-open.org/join/