Re: [PATCH 3/4] content: Introduce driver/device auxiliary notifications for MMIO

[Usama Arif <usama.arif@bytedance.com>]

On 04/04/2022 13:36, Stefan Hajnoczi wrote:
> On Wed, Mar 30, 2022 at 04:26:58PM +0100, Usama Arif wrote:
> > @@ -2074,6 +2077,16 @@ \subsection{MMIO Device Register Layout}\label{sec:Virtio Transport Options / Vi
> >       apply to the queue selected by writing to \field{QueueSel}.
> >     }
> >     \hline
> > +  \mmioreg{DeviceAuxiliaryNotification}{Device Auxiliary Notifier}{0xd0}{W}{
> > +    Writing a value to this register triggers a
> > +    notification to the device. The value written to this register is
> > +    interpreted as a device auxiliary notification index. The mapping of device auxiliary notifications to
> > +    device auxiliary notification indices is device-specific. One possible mapping would be
> > +    to use the integers 0 to N-1 as the device auxiliary notification indices for a total of
> > +    N device auxiliary notifications. The total number of device auxiliary notifications exposed by the device is
> > +    also device-specific.
> > +  }
>
> Hmm...this is slightly different from the PCI transport's approach.
> There each dev aux notification has its own hardware register. That has
> advantages:
> 1. The notification may carry a value with it because the driver writes
>     a value to the register and the device can interpret that value. That
>     can be used as a performance optimization to reduce the number of
>     hardware register writes performed by the driver or DMA transfers
>     initiated by the device.
> 2. The dev aux notification registers can be laid out on separate memory
>     pages. This is useful in passthrough scenarios like nested
>     virtualization or slicing up a device in software and passing a
>     subset to an untrusted process. The MMU only offers page-level
>     protection, so MMIO registers that are colocated within the same
>     memory page cannot be selectively passed through to untrusted
>     software.
>
> It would be nice if PCI and MMIO exposed dev aux notifications in a
> similar way.
>
> I haven't followed the history of virtio-mmio closely, so I'm not sure
> if the same approach that was used for PCI is feasible here. Maybe
> someone else can comment?
>
> Stefan


Agree with the points above. I have split the mmio implementation into 2 
registers in v2. DeviceAuxNotificationIndex to indicate the device 
auxiliary notification index and DeviceAuxNotificationData writing to 
which actually does the notification and carries the data which is 
device specific.