Re: [virtio-dev] RE: virtq configuration over PCI

[Paolo Bonzini <pbonzini@redhat.com>]

On 19/07/2017 19:56, Lior Narkis wrote:
>>> I would like to understand if I captured it right, and if so,
>>> understand how it is guaranteed today with a SW implementation of a
>>> device.

BAR writes are never pipelined or reordered.  This is probably not
helpful for HW implementations, sorry. :)

>> Guest performing queue write:
>> 1. Single write PCIe transaction with queue_select and other fields (e.g.
>> desc_addr, avail_addr) set
>> 	1.1. HW should be able to process transaction before next one (most
>> likely pipelined writes)
>> 		- however if it can't, it would just backpressure the PCIe hard
>> IP in the HW
> 
> I would not recommend back-pressuring the PCI, it will have system wide effects.
> 
> The point is not how many transactions, but the behavior of this
> special mechanism. Since there is no feedback mechanism that the
> driver can check if this configuration is completed or not, this
> memory is special. It cannot be just plain memory that stores the
> writes and responses to the reads, but needs to have a MUX logic.
Yes, queue_select needs some kind of bypass if writes are pipelined.  (I
mean bypass as in processor architecture terms, since that's the only
kind of HW that they teach you at school...).

> This is similar to the problem with doorbell(notify) writes, when a> queue numbers are written to a specific location.

I think it's different, as doorbell writes should not have a subsequent
dependency on other writes.  The doorbell is mostly a command register
(though it can be read).  The NVMe doorbells are per-queue, but I think
the completion queue doorbells do carry a "command" in the datum that
it's written to it, because you have to generate an interrupt if the
tail doesn't match the written value.

Paolo