[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] Re: Re: [virtio-comment] [PROPOSAL] Virtio Over Fabrics(TCP/RDMA)
On Tue, 25 Apr 2023 14:36:04 +0800, Jason Wang <jasowang@redhat.com> wrote: > On Mon, Apr 24, 2023 at 9:38âPM zhenwei pi <pizhenwei@bytedance.com> wrote: > > > > > > > > On 4/24/23 11:40, Jason Wang wrote: > > > On Sun, Apr 23, 2023 at 7:31âPM zhenwei pi <pizhenwei@bytedance.com> wrote: > > >> > > >> Hi, > > >> > > >> In the past years, virtio supports lots of device specifications by > > >> PCI/MMIO/CCW. These devices work fine in the virtualization environment, > > >> and we have a chance to support virtio device family for the > > >> container/host scenario. > > > > > > PCI can work for containers for sure (or does it meet any issue like > > > scalability?). It's better to describe what problems you met and why > > > you choose this way to solve it. > > > > > > It's better to compare this with > > > > > > 1) hiding the fabrics details via DPU > > > 2) vDPA > > > > > Hi, > > > > Sorry, I missed this part. "Network defined peripheral devices of virtio > > family" is the main purpose of this proposal, > > This can be achieved by either DPU or vDPA. I agree this. So I didn't understand what the meaning of this realization. Although I am also very excited to this idea, this broaden the possibility of virtio. But, I still really want to know what the meaning of this idea is, better performance? Or can achieve some situations that we cannot achieved now. > I think the advantages is, > if we standardize this in the spec, it avoids vendor specific > protocol. Sorry, I dont got this. Thanks. > > > this allows us to use many > > types of remote resources which are provided by virtio target. > > > > From the point of my view, there are 3 cases: > > 1, Host/container scenario. For example, host kernel connects a virtio > > target block service, maps it as a vdx(virtio-blk) device(used by > > Map-Reduce service which needs a fast/large size disk). The host kernel > > also connects a virtio target crypto service, maps it as virtio crypto > > device(used by nginx to accelarate HTTPS). And so on. > > > > +----------+ +----------+ +----------+ > > |Map-Reduce| | nginx | ... | processes| > > +----------+ +----------+ +----------+ > > ------------------------------------------------------------ > > Host | | | > > Kernel +-------+ +-------+ +-------+ > > | ext4 | | LKCF | | HWRNG | > > +-------+ +-------+ +-------+ > > | | | > > +-------+ +-------+ +-------+ > > | vdx | |vCrypto| | vRNG | > > +-------+ +-------+ +-------+ > > | | | > > | +--------+ | > > +---------->|TCP/RDMA|<------------+ > > +--------+ > > | > > +------+ > > |NIC/IB| > > +------+ > > | +-------------+ > > +--------------------->|virtio target| > > +-------------+ > > > > 2, Typical virtualization environment. The workloads run in a guest, and > > QEMU handles virtio-pci(or MMIO), and forwards requests to target. > > +----------+ +----------+ +----------+ > > |Map-Reduce| | nginx | ... | processes| > > +----------+ +----------+ +----------+ > > ------------------------------------------------------------ > > Guest | | | > > Kernel +-------+ +-------+ +-------+ > > | ext4 | | LKCF | | HWRNG | > > +-------+ +-------+ +-------+ > > | | | > > +-------+ +-------+ +-------+ > > | vdx | |vCrypto| | vRNG | > > +-------+ +-------+ +-------+ > > | | | > > PCI -------------------------------------------------------- > > | > > QEMU +--------------+ > > |virtio backend| > > +--------------+ > > | > > +------+ > > |NIC/IB| > > +------+ > > | +-------------+ > > +--------------------->|virtio target| > > +-------------+ > > > > So it's the job of the Qemu to do the translation from virtqueue to packet here? > > > 3, SmartNIC/DPU/vDPA environment. It's possible to convert virtio-pci > > request to virtio-of request by hardware, and forward request to virtio > > target directly. > > +----------+ +----------+ +----------+ > > |Map-Reduce| | nginx | ... | processes| > > +----------+ +----------+ +----------+ > > ------------------------------------------------------------ > > Host | | | > > Kernel +-------+ +-------+ +-------+ > > | ext4 | | LKCF | | HWRNG | > > +-------+ +-------+ +-------+ > > | | | > > +-------+ +-------+ +-------+ > > | vdx | |vCrypto| | vRNG | > > +-------+ +-------+ +-------+ > > | | | > > PCI -------------------------------------------------------- > > | > > SmartNIC +---------------+ > > |virtio HW queue| > > +---------------+ > > | > > +------+ > > |NIC/IB| > > +------+ > > | +-------------+ > > +--------------------->|virtio target| > > +-------------+ > > > > >> > > >> - Theory > > >> "Virtio Over Fabrics" aims at "reuse virtio device specifications", and > > >> provides network defined peripheral devices. > > >> And this protocol also could be used in virtualization environment, > > >> typically hypervisor(or vhost-user process) handles request from virtio > > >> PCI/MMIO/CCW, remaps request and forwards to target by fabrics. > > > > > > This requires meditation in the datapath, isn't it? > > > > > >> > > >> - Protocol > > >> The detail protocol definition see: > > >> https://github.com/pizhenwei/linux/blob/virtio-of-github/include/uapi/linux/virtio_of.h > > > > > > I'd say a RFC patch for virtio spec is more suitable than the codes. > > > > > > > OK. I'll send a RFC patch for virtio spec later if this proposal is > > acceptable. > > Well, I think we need to have an RFC first to know if it is acceptable or not. > > > > > [...] > > > > > > > > A quick glance at the code told me it's a mediation layer that convert > > > descriptors in the vring to the fabric specific packet. This is the > > > vDPA way. > > > > > > If we agree virtio of fabic is useful, we need invent facilities to > > > allow building packet directly without bothering the virtqueue (the > > > API is layout independent anyhow). > > > > > > Thanks > > > > > > > This code describes the case 1[Host/container scenario], also proves > > this case works. > > Create a virtqueue in the virtio fabric module, also emulate a > > "virtqueue backend" here, when uplayer kicks vring, the "backend" gets > > notified and builds packet to TCP/RDMA. > > In this case, it won't perform good. Since it still use virtqueue > which is unnecessary in the datapath for fabric. > > Thanks > > > > > [...] > > > > -- > > zhenwei pi > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org > For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]