[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: RFC: Doorbell suppression, packed-ring mode and hardware offload
All,I'd like to propose a small extension to the packed virtqueue mode.Â My proposal is to add an offset/wrap field, written by the driver, indicating how many available descriptors have been added to the ring.
The reason for wanting this is to improve performance of hardware devices.Â Because of high read latency over a PCIe bus, it is important for hardware devices to read multiple ring entries in parallel.Â It is desirable to know how many descriptors are available prior to issuing these reads, else you risk fetching descriptors that are not yet available.Â As well as wasting bus bandwidth this adds complexity.
I'd previously hoped that VIRTIO_F_NOTIFICATION_DATA would solve this problem, but we still have a problem.Â If you rely on doorbells to tell you how many descriptors are available, then you have to keep doorbells enabled at all times.Â This can result in a very high rate of doorbells with some drivers, which can become a severe bottleneck (because x86 CPUs can't emit MMIOs at very high rates).
The proposed offset/wrap field allows devices to disable doorbells when appropriate, and determine the latest fill level via a PCIe read.
I suggest the best place to put this would be in the driver area, immediately after the event suppression structure.
Presumably we would like this to be an optional feature, as implementations of packed mode already exist in the wild.Â How about VIRTIO_F_RING_PACKED_AVAIL_IDX?
If I prepare a patch to the spec is there still time to get this into v1.1? Best, David -- David Riddoch <firstname.lastname@example.org> -- Chief Architect, Solarflare