Re: [virtio-comment] [PATCH v3 1/1] virtio-ism: introduce new device virtio-ism

[Halil Pasic <pasic@linux.ibm.com>]

On Thu,  9 Feb 2023 11:30:56 +0800
Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote:

> +\subsection{Device configuration layout}\label{sec:Device Types / ISM Device / Device configuration layout}
> +
> +\begin{lstlisting}
> +struct virtio_ism_config {
> +	le128 cdid;
> +	le64 devid;
> +	le64 chunk_size;
> +	le64 notify_size;
> +};
> +\end{lstlisting}
> +
> +\begin{description}
> +    \item[\field{cdid}] This is used to identify the communication domain. Only
> +        ism devices with the same \field{cdid} can communicate. A \field{cdid}
> +        is world-wide unique in a sense that there not be another communication
> +        domain with the same \field{cdid}.
> +
> +    \item[\field{devid}] This is used to identify the ism device in the single
> +        communication domain.
> +
> +    \item[\field{chunk_size}] This is the size of the ism chunk. The device
> +        memory is divided into multiple chunks. Every ism chunk has the same
> +        size.
> +
> +    \item[\field{notify_size}] This is the size of the ism notify-address. The
> +        notify-address is used to notify the device that the content of the
> +        ism region has been updated.
> +
> +\end{description}
> +
> +\devicenormative{\subsubsection}{Device configuration layout}{Device Types / ISM Device / Device configuration layout}
> +
> +The device MUST ensure that the \field{cdid} of the device on the same
> +communication domain is same. The \field{cdid} MUST be a version 4 UUID as
> +specified by \hyperref[intro:rfc4122]{[RFC4122]}.
> +
> +In the single communication domain, the device MUST ensure that the \field{devid}
> +is unique.
> +


Hi Xuan Zhou!

My understanding is the following: you goal for virtio-ism is that
it should be suitable for usage with SMC-D (much like the original ISM
device). Is that right?

If yes, then let us have a look at the following example. We have
two guests sitting on the same hypervisor: A and B. Both of the
guests have an rdma capable interface, a virtio-ism device and
traditional ISM device. So they could talk over SMC-R, SMC-D via
virtio and SMC-D via (PCI-)ISM. How would the CLC Proposal message
look like?

Where I am going with this? Either you need a novel way to
discover peers (probably before the usual way is employed)
or (probably preferably) you need to make this part of the
CLC stuff. What are your ideas with regards to this? How is
it supposed to work?

To get back to the things proposed here: the cdid is IMHO
a nice thing, and is functionally corresponding to the
(S)EID. But it is 16 byte wide, and I have no idea how
is it supposed to be used in the CLC handshake.

If this is really supposed to work with SMC and not just take
inspiration from it, I would like some insight from our
SMC experts (they are already on copy).

Regards,
Halil