[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-dev] Dirty Page Tracking (DPT)
On 2020/4/13 äå9:30, Rob Miller wrote:
On Mon, Apr 13, 2020 at 8:16 AM Eugenio Perez Martin <email@example.com <mailto:firstname.lastname@example.org>> wrote:On Fri, Apr 10, 2020 at 4:40 AM Jason Wang <email@example.com <mailto:firstname.lastname@example.org>> wrote: > > > On 2020/4/10 äå5:06, Michael S. Tsirkin wrote: > > On Tue, Apr 07, 2020 at 11:52:46AM +0200, Eugenio Perez Martin wrote: > >> Hi! > >> > >> So, from the previous mails, it seems that monitoring the used ring > >> (and the packed descriptors) is a good first step in that direction, > >> as DPDK did. This way, the device does not need to worry about the > >> dirty page tracking using a bitmap and the PCI writes limitation, and > >> we can evaluate later the proposed alternatives: > >> * Alternate used descriptors in packed. > >> * vDPA interface for vDPA devices in a convenient format. > >> > >> Any thoughts? Do you think that we should start with another way? > >> > >> Thanks! > > I am concerned that with software in data path, we'll hit RX queue > > underruns, won't we? > > > Do you mean it will lose some performance? If yes, I think so. > > > > Two ways to avoid underruns: > > - dirty page tracking > > - page faults > > > It looks to me this will lead even worse performance than software path? > There will be lots of page faults during RX. > > Another direction is to track dirty pages via IOMMU. E.g recent Intel > IOMMU has EA and D bit which could be used for tracking pages wrote by > devices but not CPU. > So this could be added on top of the dirty tracking mechanism, isn't? or would it be easier to start another-way around, and to start using modern IOMMU and then extend to old generic code? > > > > > I'm working on a proposal for page faults now. > > > I guess it's better to have a transport independent method. > > > > If someone wants > > to work on dirty tracking in addition, that's also an option. > > > I remember Rob mention some challenges of implementing dirty bitmap, I > wonder something like queue based interface would be better (similar to > Peter did for KVM)? > I think that the main challenge was to write bits instead of writes using PCI bus. From a conversation with Juan, another solution could be to do a byte map DPT, where a byte represents a page, not a bit. While I find this solution simpler, I'm not sure about the performance implications of track this way (memory/TLB pressure, etc). Rob and Juan, please correct me if I'm wrong or missed something.
It would lead large working set, but the actual impact may require benchmark.
RJM>] You captured it correctly. We (Broadcom) discussed the idea of using a byte (being 0xFF) which effectively indicate that 64KB were dirty, this would cause potentially refreshing pages to the remote that weren'tneeded to be refreshed, slowing down the migration. I like the idea of mappinga byte to a page, and teach QEMU (or vDPA, or ...) how to interpret.
That's a way to go, I think we can try to propose an API and move the discussion upstream.
Compared to the used ring interposition, much more logic needs to be in the device driver for both alternatives (byte mapping and queue), isn't it? RJM> Yes, but this would (should) lead to best performance.
I agree, I think we can start from software support and discuss the hardware support in parallel.
> Thanks > --------------------------------------------------------------------- To unsubscribe, e-mail: email@example.com <mailto:firstname.lastname@example.org> For additional commands, e-mail: email@example.com <mailto:firstname.lastname@example.org>