Re: [Virtio-networking] Doorbell mapping of vDPA

[Jason Wang <jasowang@redhat.com>]

On 2020/4/15 äå12:20, Michael S. Tsirkin wrote:
> On Tue, Apr 14, 2020 at 01:12:51PM +0000, Vitaly Mireyno wrote:
> > > -----Original Message-----
> > > From: virtio-networking-bounces@redhat.com <virtio-networking-bounces@redhat.com> On Behalf
> > > Of Jason Wang
> > > Sent: Tuesday, 7 April, 2020 10:56
> > > To: virtio-networking@redhat.com; Virtio-Dev <virtio-dev@lists.oasis-open.org>
> > > Cc: Michael S. Tsirkin <mst@redhat.com>
> > > Subject: [Virtio-networking] Doorbell mapping of vDPA
> > >
> > > ----------------------------------------------------------------------
> > > Hi all:
> > >
> > > To get native performance of VF, we need to map doorbell to guest to avoid unnecessary vmexit. In
> > > order to do this, we will launch qemu with page-per-vq=on. This means the each doorbell register
> > > should be located at the beginning of 4K page and does not share the page with other registers. Then
> > > vDPA framework can safely map it into the guest physical address (GPA) range defined by qemu. It
> > > could be either
> > >
> > > 1) a single doorbell register that is used by all virtqueues
> > >
> > > or
> > >
> > > 2) several different per-vq doorbell registers
> > >
> > > If you decide to implement a virtio-pci register layout, need to make sure for notification structure
> > > (4.1.4.4 of virtio spec):
> > >
> > > For each virtqueue, the result
> > > ofcap.offset+queue_notify_off*notify_off_multiplier is PAGE_SIZE (e.g
> > > 4K) alignment, and the doorbeel does not share the page with other registers.
> > >
> > > And it would be better if queue_notify_off, notify_off_multiplier can be changed via firmware for
> > > extra flexibility.
> > In some cases, these conditions could not be met for a virtio-net hardware device over PCI transport.
> > queue_notify and notify_off_multiplier could not always be fully controlled by the firmware. There could be hardware limitations on flexibility degree of these parameters.
> > Specifically, the limitations I'm thinking of are:
> >   * queue_notify_off>0 and notify_off_multiplier>0
> >   * Several doorbell registers of several virtqueues share the same page (but don't share the page with other registers).
> >
> > Can this be supported in vDPA with direct doorbell mapping?
> >
> > Thanks
> There's value in being able to intercept some vqs in software
> while the rest of vqs are handled in hardware.
> E.g. that's the case for e.g. the control vq.


Good point, so in this case, the doorbell of control vq must exclusively 
own a page.

Or to facilitate the hardware design, we may introduce dedicated 
notification area for control vq?

Thanks


> > > Please check and make sure your hardware have such ability and feel free to ask if you have questions
> > > (offline if necessary).
> > >
> > > Thanks
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Virtio-networking mailing list
> > > Virtio-networking@redhat.com
> > > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_mailman_listinfo_virtio-
> > > 2Dnetworking&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=lDHJ2FW52oJ3lqqsArgFRdcevq01tbLQAw
> > > 4A_NO7xgI&m=pddgVCz-
> > > orGTuUXATJ4Dmi7vAXatG9w47AmULNC3V9A&s=5mfYWbLCjcZO8FcwDqgAc5bjE-H-
> > > 4p5TBkRZqP3uMsQ&e=