Re: [Virtio-networking] Doorbell mapping of vDPA

[Jason Wang <jasowang@redhat.com>]

On 2020/4/14 äå9:12, Vitaly Mireyno wrote:
> > -----Original Message-----
> > From: virtio-networking-bounces@redhat.com <virtio-networking-bounces@redhat.com> On Behalf
> > Of Jason Wang
> > Sent: Tuesday, 7 April, 2020 10:56
> > To: virtio-networking@redhat.com; Virtio-Dev <virtio-dev@lists.oasis-open.org>
> > Cc: Michael S. Tsirkin <mst@redhat.com>
> > Subject: [Virtio-networking] Doorbell mapping of vDPA
> >
> > ----------------------------------------------------------------------
> > Hi all:
> >
> > To get native performance of VF, we need to map doorbell to guest to avoid unnecessary vmexit. In
> > order to do this, we will launch qemu with page-per-vq=on. This means the each doorbell register
> > should be located at the beginning of 4K page and does not share the page with other registers. Then
> > vDPA framework can safely map it into the guest physical address (GPA) range defined by qemu. It
> > could be either
> >
> > 1) a single doorbell register that is used by all virtqueues
> >
> > or
> >
> > 2) several different per-vq doorbell registers
> >
> > If you decide to implement a virtio-pci register layout, need to make sure for notification structure
> > (4.1.4.4 of virtio spec):
> >
> > For each virtqueue, the result
> > ofcap.offset+queue_notify_off*notify_off_multiplier is PAGE_SIZE (e.g
> > 4K) alignment, and the doorbeel does not share the page with other registers.
> >
> > And it would be better if queue_notify_off, notify_off_multiplier can be changed via firmware for
> > extra flexibility.
> In some cases, these conditions could not be met for a virtio-net hardware device over PCI transport.
> queue_notify and notify_off_multiplier could not always be fully controlled by the firmware. There could be hardware limitations on flexibility degree of these parameters.
> Specifically, the limitations I'm thinking of are:
>   * queue_notify_off>0 and notify_off_multiplier>0
>   * Several doorbell registers of several virtqueues share the same page (but don't share the page with other registers).
>
> Can this be supported in vDPA with direct doorbell mapping?


I think this could be supported, we can introduce the API to:

1) let qemu to mmap() the notification pages to userspace
2) let qemu query the location of each notification registers, and 
decide whether or not it could be modeled by the transport. E.g in your 
case, if the transport is virtio-pci, it can be supported.

Thanks


> Thanks
>
> > Please check and make sure your hardware have such ability and feel free to ask if you have questions
> > (offline if necessary).
> >
> > Thanks
> >
> >
> >
> >
> >
> > _______________________________________________
> > Virtio-networking mailing list
> > Virtio-networking@redhat.com
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__www.redhat.com_mailman_listinfo_virtio-
> > 2Dnetworking&d=DwIGaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=lDHJ2FW52oJ3lqqsArgFRdcevq01tbLQAw
> > 4A_NO7xgI&m=pddgVCz-
> > orGTuUXATJ4Dmi7vAXatG9w47AmULNC3V9A&s=5mfYWbLCjcZO8FcwDqgAc5bjE-H-
> > 4p5TBkRZqP3uMsQ&e=