Re: [virtio-comment] Re: [PATCH v2 05/11] transport-fabrics: introduce Keyed Transmission

[zhenwei pi <pizhenwei@bytedance.com>]

On 6/1/23 00:20, Stefan Hajnoczi wrote:
> On Thu, May 04, 2023 at 04:19:04PM +0800, zhenwei pi wrote:
> > Keyed transmission is used for message oriented communication(Ex RDMA),
> > also add virtio-blk read/write 8K example.
> >
> > Signed-off-by: zhenwei pi <pizhenwei@bytedance.com>
> > ---
> >   transport-fabrics.tex | 178 ++++++++++++++++++++++++++++++++++++++++++
> >   1 file changed, 178 insertions(+)
> >
> > diff --git a/transport-fabrics.tex b/transport-fabrics.tex
> > index c02cf26..7711321 100644
> > --- a/transport-fabrics.tex
> > +++ b/transport-fabrics.tex
> > @@ -317,3 +317,181 @@ \subsubsection{Buffer Mapping Definition}\label{sec:Virtio Transport Options / V
> >                       |......|
> >                       +------+  -> 8193
> >   \end{lstlisting}
> > +
> > +\paragraph{Keyed Transmission}\label{sec:Virtio Transport Options / Virtio Over Fabrics / Transmission Protocol / Commands Definition / Keyed Transmission}
> > +Command and Segment Descriptors are transmitted in a message within a
> > +connection, and buffer is transmitted by remote memory access.  The layout in message:
>
> With RDMA it is theoretically possible to implement virtqueues without
> messages in the data path (i.e. by using something similar to vring with
> RDMA). Why did you decide to use a mixed messages + RDMA approach
> instead of a 100% RDMA approach?

Hi,

To reduce networking RTT. From my experience, a single RDMA 
message(event based) uses at least 6us.
This approach has a chance to send a command(include data segments) by 1 
networking RTT, and receive a completion(include data segments) in 1 
networking RTT. I tried to design a 100% RDMA approach(mapping a vring 
to the remote side, the remote side accesses this vring by RDMA 
READ/WRITE), but I failed to find an idea to achieve.

> > +
> > +\begin{lstlisting}
> > +CMDx contains 0 descriptor, CMDy contains (n - m + 1) descriptors:
>
> "0 descriptors"
>
> > +
> > +     +-----+     +-----++-----+     +-----+
> > + ... | CMDx| ... | CMDy||DESCm| ... |DESCn| ...
> > +     +-----+     +-----++-----+     +-----+
> > +
> > +COMPx contains 0 descriptor, COMPy contains 0 descriptor:
>
> "0 descriptors"

OK, I'll fix this in the next series. Thanks!

--
zhenwei pi