RE: [virtio-comment] RE: [virtio-dev] RE: [virtio-comment] [PATCH v2] virtio-net: support setting coalescing params for multiple vqs

[Parav Pandit <parav@nvidia.com>]

> From: Michael S. Tsirkin <mst@redhat.com>
> Sent: Tuesday, January 23, 2024 12:45 PM
> 
> On Tue, Jan 23, 2024 at 05:55:02AM +0000, Parav Pandit wrote:
> > > From: Michael S. Tsirkin <mst@redhat.com>
> > > Sent: Monday, January 22, 2024 1:06 PM
> > >
> > > On Mon, Jan 22, 2024 at 05:03:38AM +0000, Parav Pandit wrote:
> > > > > >>> The right test on Linux to do without rtnl lock which is
> > > > > >>> anyway ugly and
> > > > > >> wrong semantic to use blocking the whole netdev stack.
> > > > > >>> (in case if you used that).
> > > > > >> Do you have any good directions and attempts to remove rtnl_lock?
> > > > > >>
> > > > > > I think per device lock instead of rtnl is first step that we can start
> with.
> > > > >
> > > > Wil check internally who if someone already started working on it.
> > >
> > > I feel the issue is at the conceptual level.
> > Not for requests which are initiated by the kernel stack (non user initiated).
> 
> So how is this different? Is it basically just because tweaking coalescing in
> unexpected ways is considered mostly harmless?
> 
It is different in two ways.

1. requests are unrelated to each other (unlike the promisc/mac example).
For example, changing vq1.params have no effect on vq2.params.

2. Therefore they don't need to be blocked.

3. There is no user expecting a success/failure on this command.
If it fails, possibly the queues can be marked in error for recovery/re-enablement.


> > > Yes some drivers will take a command and just queue it for execution
> > > later, but this means that errors can not be propagated back at all.
> > > Imagine device with mac
> > > 0x123 in promisc mode. Now commands:
> > >
> > > 1- program MAC 0xabcdef
> > > 2- disable promisc mode
> > >
> > User initiated commands error can be propagated when the command
> completes.
> > Enqueuing command is at the different bottom level in the driver.
> >
> > > If command 1 fails but 2 proceeds then packets with MAC 0xabc will
> > > be dropped.
> > >
> > > Any attempts to batch arbitrary commands will have this issue - be
> > > it at driver or device level.
> > >
> > There is no suggestion to batch arbitrary commands from the driver side.
> > The suggestion is to batch VQs notification coalescing from the driver side.
> >
> > > So, here's my question: what exactly is the guest behaviour that is
> > > driving this work? Is it with a linux guest?
> > At least looks to me yes based on the partial patches which are taking rtnl
> lock on netdim's worker callbacks.
> >
> > > which commands does userspace issue that we need to send multiple vq
> > > coalescing commands?
> > None.
> >
> > >  If all you want is to send
> > > same config to all VQs then why not just use
> > > VIRTIO_NET_CTRL_NOTF_COAL_RX_SET as opposed to
> > > VIRTIO_NET_CTRL_NOTF_COAL_VQ_SET ?
> > Only kernel stack initiated VQ notification coalescing changes.
> > Since every VQ has different values, VIRTIO_NET_CTRL_NOTF_COAL_RX_SET
> is not sufficient.