Re: 答复: [Qemu-devel] Can we increase vring size over 1024?

["Michael S. Tsirkin" <mst@redhat.com>]

On Mon, Sep 05, 2016 at 03:21:44AM +0000, Lilijun (Jerry) wrote:
> Hi Stefan,
> 
> In our test case, the virtio TX queue is not big enough. The packets throughput sent by application in VM is not stable, Sometimes it become very large and exceed the service capability of the backend virtual switch, then they were dropped unfortunately.
> 
> To enlarge application and virtual switch's buffer ring may result other difficult problems, So we'd like to increase the virtio TX queue size to 4K or 8K.
> 
> Thanks.


Imagine a packet at the tail of the 8K queue.  How long will it take for
it to be transmitted?  If we do transmit this huge batch of 8K packets
in one go, how likely are we to cause jitter and latency for other
network users?

Looks like classical bufferbloat to me.

So again, it's a policy question and it does make sense to allow this in
future spec versions, but I can't say this is my top priority. You will
have to go ahead and design an interface to split s/g size and ring size
yourself if you are so inclined.

> -----邮件原件-----
> 发件人: Stefan Hajnoczi [mailto:stefanha@gmail.com] 
> 发送时间: 2016年9月2日 21:40
> 收件人: Gonglei (Arei)
> 抄送: mst@redhat.com; stefanha@redhat.com; jasowang@redhat.com; virtio-dev@lists.oasis-open.org; Huangpeng (Peter); qemu-devel@nongnu.org; kvm@vger.kernel.org; Lilijun (Jerry)
> 主题: Re: [Qemu-devel] Can we increase vring size over 1024?
> 
> On Fri, Sep 02, 2016 at 06:55:35AM +0000, Gonglei (Arei) wrote:
> > Michael, you made a presentation about the virto 1.1's new features in KVM Forum last week.
> > That's wonderful!
> > 
> > And I'd like to know can we increase vring size over 1024, such as 4096 or 8192?
> > 
> > My colleage had asked the same question in 2014, but she didn't get a 
> > definite answare, So, I want to rewake up the dissusstion about this. 
> > Becase for the virtio-crypto device, I also need to increase the vring 
> > size to get better performance and thoughput, but the Qemu side limit the thought as VIRTQUEUE_MAX_SIZE is 1024.
> > 
> >  [QA-virtio]:Why vring size is limited to 1024?
> > http://qemu.11.n7.nabble.com/QA-virtio-Why-vring-size-is-limited-to-10
> > 24-td292450.html
> > 
> > Avi Kivity said that google cloud exposed the vring size to 16k.
> 
> VIRTIO 1.0 allows the driver to choose the ring size (the device provides the maximum value), so in theory QEMU could offer large maximum values to VIRTIO 1.0 guests.  But existing drivers may use the device's maximum value (~16k) instead of choosing a reasonable smaller value (~1k), leading to much larger virtqueues in cases that don't need them.
> 
> I'd like to understand the root cause for the performance issue you see with virtio-net 1024 ring size.  Which queue is too small: rx or tx?
> 
> Are rx packets dropped because 1) the guest is not servicing the virtqueue quickly enough or 2) is the physical NIC delivering more packets than the rx virtqueue size?
> 
> In case #1 the guest needs to move packets from the rx virtqueue to a software queue as quickly as possible.  The virtqueue is not designed to be the incoming packet queue for the application.
> 
> In case #2 it may be possible to adjust code in the host to handle this gracefully.
> 
> Stefan