Re: [virtio-dev] Dirty Page Tracking (DPT)

[Jason Wang <jasowang@redhat.com>]

On 2020/4/13 äå8:15, Eugenio Perez Martin wrote:
> On Fri, Apr 10, 2020 at 4:40 AM Jason Wang <jasowang@redhat.com> wrote:
> > On 2020/4/10 äå5:06, Michael S. Tsirkin wrote:
> > > On Tue, Apr 07, 2020 at 11:52:46AM +0200, Eugenio Perez Martin wrote:
> > > > Hi!
> > > >
> > > > So, from the previous mails, it seems that monitoring the used ring
> > > > (and the packed descriptors) is a good first step in that direction,
> > > > as DPDK did. This way, the device does not need to worry about the
> > > > dirty page tracking using a bitmap and the PCI writes limitation, and
> > > > we can evaluate later the proposed alternatives:
> > > > * Alternate used descriptors in packed.
> > > > * vDPA interface for vDPA devices in a convenient format.
> > > >
> > > > Any thoughts? Do you think that we should start with another way?
> > > >
> > > > Thanks!
> > > I am concerned that with software in data path, we'll hit RX queue
> > > underruns, won't we?
> >
> > Do you mean it will lose some performance? If yes, I think so.
> >
> >
> > > Two ways to avoid underruns:
> > > - dirty page tracking
> > > - page faults
> >
> > It looks to me this will lead even worse performance than software path?
> > There will be lots of page faults during RX.
> >
> > Another direction is to track dirty pages via IOMMU. E.g recent Intel
> > IOMMU has EA and D bit which could be used for tracking pages wrote by
> > devices but not CPU.
> So this could be added on top of the dirty tracking mechanism, isn't?


Yes, it requires the support from IOMMU API and driver.


> or would it be easier to start another-way around, and to start using
> modern IOMMU and then extend to old generic code?


If my understanding is correct, even for modern IOMMU it still require a 
lot of work. So we need to start from software support and hardware 
support first.


> > > I'm working on a proposal for page faults now.
> >
> > I guess it's better to have a transport independent method.
> >
> >
> > > If someone wants
> > > to work on dirty tracking in addition, that's also an option.
> >
> > I remember Rob mention some challenges of implementing dirty bitmap, I
> > wonder something like queue based interface would be better (similar to
> > Peter did for KVM)?
> I think that the main challenge was to write bits instead of writes
> using PCI bus.


Yes, the ring interface will work one the descriptor which would be 
several bytes instead of bits.

Thanks


>  From a conversation with Juan, another solution could be to do a byte
> map DPT, where a byte represents a page, not a bit. While I find this
> solution simpler, I'm not sure about the performance implications of
> track this way (memory/TLB pressure, etc). Rob and Juan, please
> correct me if I'm wrong or missed something.
>
> Compared to the used ring interposition, much more logic needs to be
> in the device driver for both alternatives (byte mapping and queue),
> isn't it?
>
> > Thanks