Re: [PATCH v5 0/7] Introduce Virtio Over Fabrics

[zhenwei pi <pizhenwei@bytedance.com>]

Hi Stefan & Raphael,

Let's continue the discussion in this series, there is a link of 
multi-path/reconnect discussion with Raphael, please see the link:
https://lists.oasis-open.org/archives/virtio-comment/202307/msg00259.html


I think current version supports reconnect in several steps:
1, the transport layer reconnects successfully
2, virtio device initializes feature negotiation, sets status, setup VQ
3, retry VQ requests

The question is "which address to re-connect" in step 1?
a, Raphael pointed out redirection-based iSCSI implementations, this 
needs a "redirection" response in connect completion(from an agent), 
then connect to the specific address of target. (this needs 
specification change)
b, reconnect to the original address. (this may fail)
c, the initiator resolves domain name by DNS.
d, configure address list for a single initiator, the initiator retry 
one by one during reconnect until success.


There is limitation for multi-path, because one TCP or RDMA connection 
per VQ in current version. what about adding a new concept --- path?
A virtio-of device instance has one or more virtio-of path, a virtio-of 
path contains a control queue and several VQ.

On 10/23/23 18:46, zhenwei pi wrote:
> v4 -> v5:
> - 2 typo fixes.
> - use 'length' & 'in_length' in VQ completion.
>
> More discussion with Raphael Norwitz <raphael.norwitz@nutanix.com>:
> https://lists.oasis-open.org/archives/virtio-comment/202307/msg00259.html
>
> v3 -> v4:
> [PATCH v3 4/7] transport-fabrics: introduce command set
> - typo fixes: u16 -> le16, u64 -> le64
> - fix unaligned struct virtio_of_completion_get_keyed_num_descs
> - drop duplicated keepalive completion structure
>
> v2 -> v3:
> [PATCH v2 01/11] transport-fabrics: introduce Virtio Over Fabrics
> - Drop referring to PCI.
> - Drop undefined concept 'vring'.
> - Add 'Virtio-oF'(short for 'Virtio Over Fabrics'), and use this instead.
> - Add mapping between target and initiator.
> - Add Virtio-oF queue mapping.
> - Fix grammar flaws.
>
> [PATCH v2 02/11] transport-fabrics: introduce Virtio Qualified Name
> - Use 'NUL' instead of 'NULL'.
> - Add 'including the NUL character' in maximum name.
>
> [PATCH v2 03/11] transport-fabircs: introduce Segment Descriptor
> [PATCH v2 04/11] transport-fabrics: introduce Stream Transmission
> [PATCH v2 05/11] transport-fabrics: introduce Keyed Transmission
> - Merge three patches into one.
> - Rename this section to 'Virtio-oF Protocol Data Unit' instead of 'Transmission Protocol'.
> - Remove undefined 'opcode' reference.
> - Drop common 'struct virtio_of_vring_desc'
> - For stream data transfers, use a single command/completion instead of descriptors.
> - For keyed data transport, use a single descriptor instead of multi descriptors. Also support keyed descriptor count probe.
>
> [PATCH v2 06/11] transport-fabrics: introduce command set
> [PATCH v2 07/11] transport-fabrics: introduce opcodes
> [PATCH v2 08/11] transport-fabrics: introduce status of completion
> - Merge three patches into one.
> - Drop common data structures definition, define data structure for each command.
> - Drop virtio_of_op_get_num_queues, the Virtio-oF device instance would response error status on invalid virtqueue.
> - Allow Virtio-oF virtqueue to use empty connect body.
> - Allow Virtio-oF virtqueue to specify queue size on connecting.
> - Drop errno like status, define detailed status.
> - Other minor fixes.
>
> [PATCH v2 09/11] transport-fabrics: add TCP&RDMA binding
> - Rename TCP to TCP/IP to avoid ambiguity.
> - Add TLS TCP/IP transport.
> - Add RDMA opcode usage.
>
> [PATCH v2 10/11] transport-fabrics: add device initialization
> - Drop PCI reference, describe device initialization instead.
> - Fix grammar flaws.
>
> [PATCH v2 11/11] transport-fabrics: support inline data for keyed
> - Use eager buffer instead.
>
>
> v1 -> v2:
> - Suggested by Parav, split a large patch into several small patches.
> - Small changes for VQN, add "There is no strict style limitation".
> - Move *bytes* field limitation from get/set config opcode section to
>    Config Command.
>
>
> v1:
> Introduce Virtio-oF specification, include:
> - overview
> - Virtio Qualified Name
> - Segment Descriptor definition
> - Buffer Mapping definition: Stream Transmission and Keyed Transmission
> - Command set definition
> - opcode definition
> - status definition
> - transport binding: TCP and RDMA
> - device initialization
>
> Previous discussion:
> https://lists.oasis-open.org/archives/virtio-comment/202304/msg00442.html
>
> zhenwei pi (7):
>    transport-fabrics: introduce Virtio Over Fabrics overview
>    transport-fabrics: introduce Virtio-oF Qualified Name
>    transport-fabrics: introduce Virtio-oF Protocol Data Unit
>    transport-fabrics: introduce command set
>    transport-fabrics: introduce transport binding
>    transport-fabrics: add device initialization
>    transport-fabrics: introduce keyed eager buffers
>
>   content.tex           |   1 +
>   transport-fabrics.tex | 875 ++++++++++++++++++++++++++++++++++++++++++
>   2 files changed, 876 insertions(+)
>   create mode 100644 transport-fabrics.tex

--
zhenwei pi