[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-comment] Re: [virtio] [PATCH requirements 6/7] net-features: Add packet timestamp requirements
On Fri, Aug 11, 2023 at 5:28âAM Willem de Bruijn <willemdebruijn.kernel@gmail.com> wrote: > > On Thu, Aug 10, 2023 at 2:56âAM Jason Wang <jasowang@redhat.com> wrote: > > > > On Wed, Aug 9, 2023 at 4:47âPM Xuan Zhuo <xuanzhuo@linux.alibaba.com> wrote: > > > > > > On Mon, 24 Jul 2023 06:34:20 +0300, Parav Pandit <parav@nvidia.com> wrote: > > > > Add tx and rx packet timestamp requirements. > > > > > > > > Signed-off-by: Parav Pandit <parav@nvidia.com> > > > > --- > > > > net-workstream/features-1.4.md | 26 ++++++++++++++++++++++++++ > > > > 1 file changed, 26 insertions(+) > > > > > > > > diff --git a/net-workstream/features-1.4.md b/net-workstream/features-1.4.md > > > > index d228462..37820b6 100644 > > > > --- a/net-workstream/features-1.4.md > > > > +++ b/net-workstream/features-1.4.md > > > > @@ -10,6 +10,7 @@ together is desired while updating the virtio net interface. > > > > 2. Low latency tx and rx virtqueues for PCI transport > > > > 3. Virtqueue notification coalescing re-arming support > > > > 4 Virtqueue receive flow filters (RFF) > > > > +5. Device timestamp for tx and rx packets > > > > > > > > # 3. Requirements > > > > ## 3.1 Device counters > > > > @@ -280,3 +281,28 @@ struct virtio_net_rff_delete { > > > > u8 padding[2]; > > > > le32 flow_id; > > > > }; > > > > + > > > > +## 3.5 Packet timestamp > > > > +1. Device should provide transmit timestamp and receive timestamp of the packets > > > > + at per packet level when the device is enabled. > > > > +2. Device should provide the current free running clock in the least latency > > > > + possible using an MMIO register read of 64-bit to have the least jitter. Btw, register is not offered via all transports. > > > > +3. Device should provide the current frequency and the frequency unit for the > > > > + software to synchronize the reference point of software and the device using > > > > + a control vq command. > > > > + > > > > +### 3.5.1 Transmit timestamp > > > > +1. Transmit completion must contain a packet transmission timestamp when the > > > > + device is enabled for it. > > > > +2. The device should record the packet transmit timestamp in the completion at > > > > + the farthest egress point towards the network. > > > > +3. The device must provide a transmit packet timestamp in a single DMA > > > > + transaction along with the rest of the transmit completion fields. > > > > + > > > > +### 3.5.2 Receive timestamp > > > > +1. Receive completion must contain a packet reception timestamp when the device > > > > + is enabled for it. > > > > +2. The device should record the received packet timestamp at the closet ingress > > > > + point of reception from the network. > > > > +3. The device should provide a receive packet timestamp in a single DMA > > > > + transaction along with the rest of the receive completion fields. > > > > > > > > > According to the last discuss, the feature will depend on the new desc > > > structure. > > > > > > I would to know, can we introduce this to the current spec with a simple change? > > > > > > struct vring_used_elem { > > > /* Index of start of used descriptor chain. */ > > > __virtio32 id; > > > /* Total length of the descriptor chain which was used (written to) */ > > > __virtio32 len; > > > > > > + __virtio64 timestamp; > > > }; > > > > I think this could be one way and another proposal from Willem is: > > > > https://lists.linuxfoundation.org/pipermail/virtualization/2021-February/052422.html > > > > which might be tricky for TX but it's more flexible since it allows > > timestamps to be done per buffer. > > Thanks for looping me in, Jason. > > I would separate timestamping data from clock synchronization and > timestamping control path. > > > For synchronization, a single 64-bit MMIO read is not necessarily the > fastest when traversing real hardware boundaries, or sufficient. PCIe > has Precision Time Measurement (PTM) hardware logic to capture > clock measurement with less uncertainty and possibly latency, for instance. > Indeed, uncertainty is more important than raw latency. > > FWIW, we're also trying to capture similar requires for physical devices > through the Open Compute Platform NIC Core Offloads effort: > https://www.opencompute.org/w/index.php?title=Core_Offloads#Timestamping This is interesting and the whole idea of "NIC Core Offloads" looks great as well. > > Virtio defines both a virtual interface and one that can be supported > by hardware, so it's not a 1:1 match, but many subtle points probably > apply to both. Yeah, I think virito-net may try to align with that. > > > For timestamping, the difficult part is the transmit completion stamp, > asynchronous with data flow. If this is taken on the device before or > at the time of queuing the completion to the host, then it can be stored > in\ the completion descriptor. Yes, this is the proposal from Xuan actually. > But if taken at the PHY, say, it is not > uncommon for this to happen after the completion is written. This seems the requirement of the this patch which says: " The device should record the packet transmit timestamp in the completion at the farthest egress point towards the network. " It seems to imply the PHY but with a DMA interface: " The device must provide a transmit packet timestamp in a single DMA transaction along with the rest of the transmit completion fields. " Parav, can you clarify? And > And > instead a block of dedicated registers is used and the host must poll > a register. There are examples of these in the Linux device driver > directory. For virtio, it is probably sufficient to only support the > first kind. RIght, for the second, it could be tricky for a virtual environemnt. > > > A third aspect is the control path: querying and configuring hardware > timestamps (siocgstamp/siosgstamp). If hardware is capable, easiest is > to just timestamp every packet. Much hardware out there targets low > rate PTP messages. But a virtualized virtio device could timestamp > all. In which case this control API can maybe be punted on. > That could be one way. Thanks
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]