Re: Re: [PATCH v2 01/11] transport-fabrics: introduce Virtio Over Fabrics overview

[zhenwei pi <pizhenwei@bytedance.com>]

On 5/31/23 22:00, Stefan Hajnoczi wrote:
> On Thu, May 04, 2023 at 04:19:00PM +0800, zhenwei pi wrote:
> > In the past years, virtio supports lots of device specifications by
> > PCI/MMIO/CCW. These devices work fine in the virtualization environment.
> >
> > Introduce Virtio Over Fabrics transport to support "network defined
> > peripheral devices". With this transport, Many Virtio based devices
> > transparently work over fabrics. Note that the balloon device may not
> > make sense. Shared memory regions won't work.
> >
> > Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
> > ---
> >   content.tex           |  1 +
> >   transport-fabrics.tex | 31 +++++++++++++++++++++++++++++++
> >   2 files changed, 32 insertions(+)
> >   create mode 100644 transport-fabrics.tex
> >
> > diff --git a/content.tex b/content.tex
> > index cff548a..f899c3a 100644
> > --- a/content.tex
> > +++ b/content.tex
> > @@ -582,6 +582,7 @@ \chapter{Virtio Transport Options}\label{sec:Virtio Transport Options}
> >   \input{transport-pci.tex}
> >   \input{transport-mmio.tex}
> >   \input{transport-ccw.tex}
> > +\input{transport-fabrics.tex}
> >   
> >   \chapter{Device Types}\label{sec:Device Types}
> >   
> > diff --git a/transport-fabrics.tex b/transport-fabrics.tex
> > new file mode 100644
> > index 0000000..0dc031b
> > --- /dev/null
> > +++ b/transport-fabrics.tex
> > @@ -0,0 +1,31 @@
> > +\section{Virtio Over Fabrics}\label{sec:Virtio Transport Options / Virtio Over Fabrics}
> > +
> > +This section defines specification to Virtio that enables operation over other
> > +interconnects. A central goal of Virtio Over Fabrics is to maintain consistency
> > +with the PCI device, so Virtio based devices transparently work over PCI or
> > +fabrics.
>
> The reader wants to know what VIRTIO Over Fabrics is, not how it relates
> to other Transports that they may not be very familiar with.
>
> Fabrics is a Transport and any Transport is capable of supporting the
> VIRTIO device model. Therefore I don't think the stated aim should be to
> match PCI specifically. Just being a Transport is already enough. PCI is
> not special.
>
> I suggest something like:
>
>    Virtio Over Fabrics enables operation over interconnects that rely
>    primarily on message passing. Supported interconnects include TODO.
>
> > +
> > +Virtio Over Fabrics uses reliable connection to transmit data, the reliable
>
> "uses a reliable connection"
>
> > +connection betweens two rules:
>
> "connection facilitates communication between entities playing the following roles:"
>
> > +
> > +\begin{itemize}
> > +\item An initiator functions as an Virtio Over Fabrics client. An initiator
>
> "as a Virtio ..."
>
> > +typically serves the same purpose to a machine as a Virtio device, issues
> > +commands to remote side.
>
> This says that the driver talks to the initiator instead of a local
> device and the initiator forwards commands to the actual device on the
> remote side?
>
> I find this sentence confusing because I associate the initiator with
> the driver, not the device.
>
> Maybe:
>
>    The initiator sends commands from the driver to the target.
>
> > +\item A target functions as an Virtio Over Fabrics server. An target typically
>
> "A target"
>
> > +handles commands from the initiator side and responses completions.
>
> The concept of the device is missing here. For symmetry it may be good
> to say something like:
>
>    The target forwards commands to the device and sends responses back to
>    the initiator.
>
> > +\end{itemize}
> > +
> > +Virtio Over Fabrics has the following differences from the PCI based
> > +specification:
> > +
> > +\begin{itemize}
> > +\item Instead of memory sharing mechanism of virtqueue, there is a one-to-one
> > +mapping between virtqueue and the reliable connection which executes the vring
> > +data transmission.
> > +\item An additional control connection is required to execute control commands
> > +which is similar to read/write register on a PCI device.
> > +\item Virtio Over Fabrics does not define an interrupt mechanism that allows an
> > +initiator to generate a host interrupt. It is the responsibility of the host
> > +fabric interface to generate host interrupts.
> > +\end{itemize}
>
> As mentioned above, comparing against PCI requires that the reader is
> familiar with PCI. I think it would be preferrable to explain the unique
> characteristics of Virtio Over Fabrics in a self-contained way:
>
>    The basic organization of Virtio Over Fabrics is as follows:
>
>    \begin{itemize}
>    \item A reliable connection carries control commands that are not specific to a virtqueue.
>    \item Each virtqueue has its own reliable connection.
>    \item There is no interrupt mechanism since the arrival of data on the fabric already indicates when there is activity.
>    \end{itemize}
>
> Stefan

I'll drop comparing against PCI part, and fix other parts in the next 
version. Thanks!

--
zhenwei pi