Re: [virtio-dev] queue_enable vs QueueReady

[Jason Wang <jasowang@redhat.com>]

On 2020/5/28 äå10:00, Stefan Hajnoczi wrote:
> On Thu, May 28, 2020 at 09:06:36PM +0800, Jason Wang wrote:
> > Hi:
> >
> > I found ambiguity in the virtio specification:
> >
> > In PCI part, it describes the queue_enable as:
> >
> > The driver uses this to selectively prevent the device from executing
> > requests from this virtqueue. 1 - enabled; 0 - disabled.
> >
> > In MMIO part, it describes the QueueReady as:
> >
> > Writing one (0x1) to this register notifies the device that it can execute
> > requests from this virtual queue. Reading from this register returns the
> > last value written to it. Both read and write accesses apply to the queue
> > selected by writing to QueueSel.
> >
> > If I understand this correctly, they have the same meaning, but the driver
> > requirements section looks conflict:
> >
> > PCI said: The driver MUST NOT write a 0 to queue_enable.
> >
> > MMIO said:
> >
> > To stop using the queue the driver MUST write zero (0x0) to this QueueReady
> > and MUST read the value back to ensure synchronization.
> >
> > So we can't disable a queue via queue_enable but QueueReady. Any reason for
> > such inconsistency?
> I think MMIO is the outlier here. The device emulation code in QEMU
> doesn't deal with queue shutdown. That only happens during device reset,
> so it's like that a if the guest disables a virtio-mmio queue then
> something undefined will happen on the QEMU side.


Qemu's MMIO only implement the fucntion of value write and read. Its PCI 
implementation is also buggy which assumes the value is one.


>   For example, writing
> used elements back to the virtqueue after it has been disabled.


Ok, but kernel virtio-mmio driver did the following in vm_del_vq():

ÂÂÂ /* Select and deactivate the queue */
ÂÂÂ writel(index, vm_dev->base + VIRTIO_MMIO_QUEUE_SEL);
ÂÂÂ if (vm_dev->version == 1) {
ÂÂÂ ÂÂÂ writel(0, vm_dev->base + VIRTIO_MMIO_QUEUE_PFN);
ÂÂÂ } else {
ÂÂÂ ÂÂÂ writel(0, vm_dev->base + VIRTIO_MMIO_QUEUE_READY);
ÂÂÂ ÂÂÂ WARN_ON(readl(vm_dev->base + VIRTIO_MMIO_QUEUE_READY));
ÂÂÂ }

Which tries to align with the spec ...


> If the VIRTIO spec really intends to support virtqueue shutdown then the
> semantics need to be spelled out clearly: what happens to in-flight
> requests? How do devices behave that rely on multiple virtqueues during
> normal operation (e.g. virtio-vsock where you can't really have rx-only
> or tx-only)?


If we plan to do this, we need do something similar to status. That 
means driver must wait for a read of queue_enable to return 0.

For in-flight request we can do the same thing as device reset but just 
for a virtqueue probably.

Thanks


> Stefan