[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] [PATCH v4] content: Introduce VIRTIO_NET_F_STANDBY feature
On Mon, Nov 26, 2018 at 5:13 PM Sameeh Jubran <sameeh@daynix.com> wrote: > > On Thu, Nov 22, 2018 at 8:27 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > On Wed, Nov 21, 2018 at 10:04:53PM +0200, Sameeh Jubran wrote: > > > On Wed, Nov 21, 2018 at 8:41 PM Michael S. Tsirkin <mst@redhat.com> wrote: > > > > > > > > Great to see you making progress on this! > > > > Some comments below: > > > > > > > > On Wed, Nov 21, 2018 at 05:39:38PM +0200, Sameeh Jubran wrote: > > > > > I have created a setup which has two hosts (host A and host B) with X710 10G > > > > > cards connected back to back. On one host (I'll refer to this host as host A) I > > > > > have configured a bridge with the PF interface as well as vitio-net's interface > > > > > (standby) both attached to it. > > > > > > > > ... > > > > > > > > > The command line I used: > > > > > > > > > > /root/qemu/x86_64-softmmu/qemu-system-x86_64 \ > > > > > -netdev tap,id=hostnet0,script=world_bridge_standalone.sh,downscript=no,ifname= > > > > > cc17 \ > > > > > -device e1000,netdev=hostnet0,mac=56:cc:c1:01:cc:21,id=cc17 \ > > > > > > > > What's e1000 doing here? > > > > Can this be reason you can not talk to host? > > > I don't think so, the e1000 is for enabling WAN connection on the > > > guest for downloading packages and ssh connection. It is connected to > > > a separate bridge which is connected to the external interface of the > > > host. > > > > > > > > > -netdev tap,vhost=on,id=hostnet1,script=test_bridge_standalone.sh,downscript= > > > > > no,ifname=cc1_72,queues=4 \ > > > > > -device virtio-net,host_mtu=1500,netdev=hostnet1,mac=8a:f7:20:29:3b:cb,id= > > > > > cc1_72,vectors=10,mq=on,primary=cc1_71 \ > > > > > -device vfio-pci,host=65:02.1,id=cc1_71,standby=cc1_72 \ > > > > > -enable-kvm \ > > > > > -name netkvm \ > > > > > -m 3000M \ > > > > > -drive file=/dev/shm/fedora_29.qcow2,if=ide,id=drivex \ > > > > > -smp 4 \ > > > > > -vga qxl \ > > > > > -spice port=6110,disable-ticketing \ > > > > > -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x7 \ > > > > > -chardev spicevmc,name=vdagent,id=vdagent \ > > > > > -device virtserialport,nr=1,bus=virtio-serial0.0,chardev=vdagent,name= > > > > > com.redhat.spice.0 \ > > > > > -chardev socket,path=/tmp/qga.sock,server,nowait,id=qga0 \ > > > > > -device virtio-serial \ > > > > > -device virtserialport,chardev=qga0,name=org.qemu.guest_agent.0 \ > > > > > -monitor stdio > > > > > > > > > > > > ... > > > > > > > > > Since I couldn't ping from VM to host B, I did an iperf test between the VM and > > > > > host A with the feature enabled and during the test I have unplugged the sriov > > > > > device, the device was unplugged successfully and no drops where observed as > > > > > you can see in the results below: > > > > > > > > > > [root@dhcp156-44 ~]# ifconfig > > > > > > > > Well I suspect this won't tell you anything, this shows packet drops at > > > > the hardware level. When e.g. link is down linux won't send any packets > > > > out. The simplest test is to monitor latency and throughput and see that > > > > while it is lower for the duration of migration, there are no huge > > > > spikes around the switch. > > > Oh, okay will do that. > > > > > > I have noticed some nasty lag when I tried to ssh to the VM using the > > > failover interface while I didn't experience that with the e1000. > > > Sridhar Any idea what might be the cause? > > > > Try tcpdump? > I have investigated this and this is what I have so far, maybe you can > help me with some insights to figure what's going on. > The setup is as follows: > > > > |_VM_| > __||___ > |host A|----X710---------back-to-back--------X710---|host B| > > _______________________________________________________________________ > - On the host A: > > I have the following interfaces attached to the "test_br0" bridge: > > virtio-net's netdev, cc1_72 > X710 device PF interfaces: ens2f0 and ens2f1 (only ens2f0 is connected > in the back to back setup) > > The bridge has the mac address of the PF ens2f0 and ip : 192.168.1.117 > _______________________________________________________________________ > - On the host B: > > I have the following interfaces attached to the "test_br0" bridge: > > X710 device PF interfaces: ens2f0 and ens2f1 (only ens2f0 is connected > in the back to back setup) > > The bridge has the mac address of the PF ens2f0 and ip : 192.168.1.118 > _______________________________________________________________________ > - On the VM: > The failover interface has the ip: 192.168.1.17 > _______________________________________________________________________ > > I can successfully ping 118 from 17. (host B from the VM), however I > can't see the ICMP requests on host A anywhere! > I can see them inside host B on ens2f0, I can see them in the VM on > the failover interface but not on Host A. > Not on the brdige (test_br0) as I would expect, not on the ens2f0 > interface, not co cc1_72 (virtio-net) interface and of-course not on > the world interface. > This leads me to think that the icmp requests are send on the "vf" > interface which I cant see on the host. The thing that further > confirms my theory is when > I use device_del to unplug the primary interface, the ping get > disconnected. Using tcpdump I can see that the ping requests arrive to > host B and there is a > suitable ping reply, however the reply is not present on Host A or the > VM anywhere, moreover, when the primary gets disconnected I start > seeing the ping > requests on Host A on the "test_br0" and "ens2f0". > > Liran do you think this is related to the mac vtables and vfs issue > that you've mentioned on the monthly meeting? > > Update: I have just set the vf mac's address to 0 (ip link set ens2f0 vf 1 mac 00:00:00:00:00:00) after unplugging it (the primary device) and the pings started working again on the failover interface. So it seems like the frames were arriving to the vf on the host. > > > > > > > > > > -- > > > > MST > > > > > > > > > > > > -- > > > Respectfully, > > > Sameeh Jubran > > > Linkedin > > > Software Engineer @ Daynix. > > > > -- > Respectfully, > Sameeh Jubran > Linkedin > Software Engineer @ Daynix. -- Respectfully, Sameeh Jubran Linkedin Software Engineer @ Daynix.
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]