Re: [Qemu-devel] [virtio-dev] Re: [virtio-dev] Re: [PATCH v2 00/16] Vhost-pci for inter-VM communication

[Wei Wang <wei.w.wang@intel.com>]

On 05/24/2017 11:24 AM, Jason Wang wrote:
> On 2017年05月23日 18:48, Wei Wang wrote:
> > On 05/23/2017 02:32 PM, Jason Wang wrote:
> > > On 2017年05月23日 13:47, Wei Wang wrote:
> > > > On 05/23/2017 10:08 AM, Jason Wang wrote:
> > > > > On 2017年05月22日 19:46, Wang, Wei W wrote:
> > > > > > On Monday, May 22, 2017 10:28 AM, Jason Wang wrote:
> > > > > > > On 2017年05月19日 23:33, Stefan Hajnoczi wrote:
> > > > > > > > On Fri, May 19, 2017 at 11:10:33AM +0800, Jason Wang wrote:
> > > > > > > > > On 2017年05月18日 11:03, Wei Wang wrote:
> > > > > > > > > > On 05/17/2017 02:22 PM, Jason Wang wrote:
> > > > > > > > > > > On 2017年05月17日 14:16, Jason Wang wrote:
> > > > > > > > > > > > On 2017年05月16日 15:12, Wei Wang wrote:
> > > > > > > > > > > > > > Hi:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Care to post the driver codes too?
> > > > > > > > > > > > > OK. It may take some time to clean up the driver code 
> > > > > > > > > > > > > before post
> > > > > > > > > > > > > it out. You can first have a check of the draft at the 
> > > > > > > > > > > > > repo here:
> > > > > > > > > > > > > https://github.com/wei-w-wang/vhost-pci-driver
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best,
> > > > > > > > > > > > > Wei
> > > > > > > > > > > > Interesting, looks like there's one copy on tx side. We 
> > > > > > > > > > > > used to
> > > > > > > > > > > > have zerocopy support for tun for VM2VM traffic. Could you 
> > > > > > > > > > > > please
> > > > > > > > > > > > try to compare it with your vhost-pci-net by:
> > > > > > > > > > We can analyze from the whole data path - from VM1's network 
> > > > > > > > > > stack
> > > > > > > > > > to send packets -> VM2's network stack to receive packets. The
> > > > > > > > > > number of copies are actually the same for both.
> > > > > > > > > That's why I'm asking you to compare the performance. The only 
> > > > > > > > > reason
> > > > > > > > > for vhost-pci is performance. You should prove it.
> > > > > > > > There is another reason for vhost-pci besides maximum performance:
> > > > > > > >
> > > > > > > > vhost-pci makes it possible for end-users to run networking or 
> > > > > > > > storage
> > > > > > > > appliances in compute clouds.  Cloud providers do not allow 
> > > > > > > > end-users
> > > > > > > > to run custom vhost-user processes on the host so you need 
> > > > > > > > vhost-pci.
> > > > > > > >
> > > > > > > > Stefan
> > > > > > > Then it has non NFV use cases and the question goes back to the 
> > > > > > > performance
> > > > > > > comparing between vhost-pci and zerocopy vhost_net. If it does 
> > > > > > > not perform
> > > > > > > better, it was less interesting at least in this case.
> > > > > > Probably I can share what we got about vhost-pci and vhost-user:
> > > > > > https://github.com/wei-w-wang/vhost-pci-discussion/blob/master/vhost_pci_vs_vhost_user.pdf 
> > > > > >
> > > > > > Right now, I don’t have the environment to add the vhost_net test.
> > > > >
> > > > > Thanks, the number looks good. But I have some questions:
> > > > >
> > > > > - Is the number measured through your vhost-pci kernel driver code?
> > > >
> > > > Yes, the kernel driver code.
> > >
> > > Interesting, in the above link, "l2fwd" was used in vhost-pci 
> > > testing. I want to know more about the test configuration: If l2fwd 
> > > is the one that dpdk had, want to know how can you make it work for 
> > > kernel driver. (Maybe packet socket I think?) If not, want to know 
> > > how do you configure it (e.g through bridge or act_mirred or 
> > > others). And in OVS dpdk, is dpdk l2fwd + pmd used in the testing?
> >
> > Oh, that l2fwd is a kernel module from OPNFV vsperf
> > (http://artifacts.opnfv.org/vswitchperf/docs/userguide/quickstart.html)
> > For both legacy and vhost-pci cases, they use the same l2fwd module.
> > No bridge is used, the module already works at L2 to forward packets
> > between two net devices.
>
> Thanks for the pointer. Just to confirm, I think virtio-net kernel 
> driver is used in OVS-dpdk test?

Yes. In both cases, the guests are using kernel drivers.

> Another question is, can we manage to remove the copy in tx? If not, 
> is it a limitation of virtio protocol?

No, we can't. Use this example, VM1's Vhost-pci<->virtio-net of VM2, VM1 
sees VM2's memory, but
VM2 only sees its own memory.
What this copy achieves is to get data from VM1's memory to VM2's 
memory, so that VM2 can deliver it's
own memory to its network stack.

Best,
Wei