RE: [RFC PATCH v3 1/4] virtio-rtc: Add initial device specification

[Parav Pandit <parav@nvidia.com>]

Hi Peter,

> From: Peter Hilber <peter.hilber@opensynergy.com>
> Sent: Monday, December 18, 2023 12:13 PM
> 
> This is a draft for a Real-Time Clock (RTC) device. The virtio-rtc device
> provides information about current time through one or more clocks.
> 
> The normative statements for this device follow in the next patch.
> 
> For this device, there is an RFC Linux kernel driver which is being
> upstreamed, and a proprietary device implementation.
> 
> Open Questions
> ==============
> 
> Arm Generic Timer Clock Types
> -----------------------------
> 
> This spec version still distinguishes the Arm Generic Timer virtual and
> physical counters:
> 
> 	/* Arm Generic Timer CNTVCT_EL0, Counter-timer Virtual Count
> register */
> 	#define VIRTIO_RTC_COUNTER_ARM_VIRT 0
> 	/* Arm Generic Timer CNTPCT_EL0, Counter-timer Physical Count
> register */
> 	#define VIRTIO_RTC_COUNTER_ARM_PHYS 1
> 
> But there might be no benefit distinguishing them, as per a discussion on the
> RFC Linux kernel driver [4].
> 
> So unless somebody identifies some use in distinguishing Arm virtual and
> physical counters, this distinction will be dropped in v4.
> 
> virtio-rtc Functionality for Net Device
> ---------------------------------------
> 
> There is interest into sharing virtio-rtc functionality with the virtio-net device
> [6]. It is proposed to encapsulate the requests resp.
> responses in the pseudo-fields command-specific-data resp.
> command-specific-result.
> 
> This handles natural alignment of the payload in a way which is compatible
> with
> 
> - existing virtio-net controlq commands, and
> 
> - with the standalone virtio-rtc messages.
> 
> Additional Information
> ======================
> 
> The spec does not specify how a driver should interpret clock readings, esp.
> also not how to perform clock synchronization.
> 
> The draft uses the "Timer/Clock" device id which is already part of the
> specification. This device id was registered a long time ago and should be
> unused according to the author's information. The name "RTC" was
> determined to be the best for a device which focuses on current time, but is
> up to discussion.
> 
> Any feedback is appreciated very much.
> 
> Changelog
> =========
> 
> v3:
> 
> - Address comments from Parav Pandit.
> - Split off normative requirements into a second commit [2].
> - Merge readq and controlq into requestq [3].
> - Don't guard cross-timestamping with feature bit [3].
> - Pad request headers to 64 bit [2].
> - Rename Virtio status codes to match UNIX error names [2].
> - Avoid Virtio status code clashes with net controlq ack values.
> - Reword to refer more to "requests", rather than "messages" [2].
> - Rephrase some sentences [2].
> - Use integer data types without "__" prefixes [2].
> - Reduce clock id width to 16 bits [5].
> - Make VIRTIO_RTC_FLAG_CROSS_CAP a bit mask rather than a bit index.
> 
> v2:
> 
> - Address comments from Cornelia Huck.
> - Add VIRTIO_RTC_M_CROSS_CAP message [1].
> - Fix various minor issues and improve wording [1].
> - Add several clarifications regarding device error statuses.
> 
> [1] https://lists.oasis-open.org/archives/virtio-
> comment/202304/msg00523.html
> [2] https://lore.kernel.org/virtio-comment/b59a7dda-06fe-cff9-df61-
> b90aa4e50836@opensynergy.com/t/#mffb93800fea11d6dda9e151078abedd
> 6ff1c0f1e
> [3] https://lore.kernel.org/virtio-comment/b59a7dda-06fe-cff9-df61-
> b90aa4e50836@opensynergy.com/t/#m94efd0aa9b9c2b96a246b79ef8bfc3bf
> 64ebe791
> [4] https://lore.kernel.org/lkml/20230630171052.985577-1-
> peter.hilber@opensynergy.com/T/#m65fa1d715933360498c4e33d7225e4220
> 215a9d6
> [5] https://lore.kernel.org/virtio-comment/b59a7dda-06fe-cff9-df61-
> b90aa4e50836@opensynergy.com/t/#mf00ce330228c28556d735eb9597469
> 048c5d8b62
> [6] https://lore.kernel.org/virtio-comment/b59a7dda-06fe-cff9-df61-
> b90aa4e50836@opensynergy.com/t/#mf00ce330228c28556d735eb9597469
> 048c5d8b62
> 
> Signed-off-by: Peter Hilber <peter.hilber@opensynergy.com>
> ---
>  content.tex                      |   3 +-
>  device-types/rtc/description.tex | 328 +++++++++++++++++++++++++++++++
>  2 files changed, 330 insertions(+), 1 deletion(-)  create mode 100644 device-
> types/rtc/description.tex
> 
> diff --git a/content.tex b/content.tex
> index 98b9c319a666..f2b534ce97e6 100644
> --- a/content.tex
> +++ b/content.tex
> @@ -683,7 +683,7 @@ \chapter{Device Types}\label{sec:Device Types}
> \hline
>  16         &   GPU device       \\
>  \hline
> -17         &   Timer/Clock device \\
> +17         &   RTC (Timer/Clock) device \\
>  \hline
>  18         &   Input device \\
>  \hline
> @@ -769,6 +769,7 @@ \chapter{Device Types}\label{sec:Device Types}
> \input{device-types/scmi/description.tex}
>  \input{device-types/gpio/description.tex}
>  \input{device-types/pmem/description.tex}
> +\input{device-types/rtc/description.tex}
> 
>  \chapter{Reserved Feature Bits}\label{sec:Reserved Feature Bits}
> 
> diff --git a/device-types/rtc/description.tex b/device-types/rtc/description.tex
> new file mode 100644
> index 000000000000..abfa2206894e
> --- /dev/null
> +++ b/device-types/rtc/description.tex
> @@ -0,0 +1,328 @@
> +\section{RTC Device}\label{sec:Device Types / RTC Device}
> +
> +The RTC (Real Time Clock) device provides information about current
> +time. The device can provide different clocks, e.g.\ for the UTC or TAI
> +time standards, or for physical time elapsed since some past epoch. The
> +driver can read the clocks with simple or more accurate methods.
> +
> +\subsection{Device ID}\label{sec:Device Types / RTC Device / Device ID}
> +
> +17
> +
> +\subsection{Virtqueues}\label{sec:Device Types / RTC Device /
> +Virtqueues}
> +
> +\begin{description}
> +\item[0] requestq
> +\end{description}
> +
> +The driver enqueues requests to the requestq.
> +
> +\subsection{Feature bits}\label{sec:Device Types / RTC Device / Feature
> +bits}
> +
> +No device-specific feature bits are defined yet.
> +
> +\subsection{Device configuration layout}\label{sec:Device Types / RTC
> +Device / Device configuration layout}
> +
> +There is no configuration data for the device.
> +
> +\subsection{Device Initialization}\label{sec:Device Types / RTC Device
> +/ Device Initialization}
> +
> +The device determines the set of clocks. The device can provide zero or
> +more clocks.
> +
Now that we bring the generic basic facilities to define device capabilities and resources in [1],
Num_clocks can be structured a clock resources.

Please refer to max_limits in [1] as capabilities.
[1] https://lists.oasis-open.org/archives/virtio-comment/202401/msg00095.html

We should bring a rtc capabilities.

> +\subsection{Device Operation}\label{sec:Device Types / RTC Device /
> +Device Operation}
> +
> +For the requestq, the driver sends a message with a request, and
> +receives the response in the device-writable part of the message. The
> +requestq uses common request and response headers.
> +
Now without defining any rtc specific structure, we benefit from generic structure.
Only two things needed.
1. define cap identifier
2. define cap structure

> +\begin{lstlisting}
> +/* common request header */
> +struct virtio_rtc_req_head {
> +	le16 msg_type;
> +	u8 reserved[6];
> +};
> +
> +/* common response header */
> +struct virtio_rtc_resp_head {
> +	u8 status;
> +	u8 reserved[7];
> +};
> +\end{lstlisting}
> +
> +The \field{msg_type} field identifies the message type.
> +
> +The \field{status} field indicates whether the device successfully
> +executed the request. The device sets the \field{status} field to one
> +of the following values:
> +
> +\begin{lstlisting}
> +#define VIRTIO_RTC_S_OK 0
> +#define VIRTIO_RTC_S_EOPNOTSUPP 2
> +#define VIRTIO_RTC_S_ENODEV 3
> +#define VIRTIO_RTC_S_EINVAL 4
> +#define VIRTIO_RTC_S_EIO 5
> +\end{lstlisting}
> +
> +VIRTIO_RTC_S_OK indicates that the device successfully executed the
> +request.
> +
> +If \field{status} is not VIRTIO_RTC_S_OK, the value of other response
> +fields is undefined.
> +
> +VIRTIO_RTC_S_EOPNOTSUPP indicates that the device could not execute
> the
> +specific request due to an implementation limitation. The device also
> +returns status VIRTIO_RTC_S_EOPNOTSUPP for requests with unknown
> values
> +in the fields \field{msg_type} or \field{hw_counter}.
> +
> +VIRTIO_RTC_S_ENODEV indicates that the \field{clock_id} field value
> +supplied with the request does not identify a clock.
> +
> +VIRTIO_RTC_S_EINVAL indicates that
> +
> +\begin{itemize}
> +\item the driver request values are not allowed by the specification or
> +\item the device read-only part of the message, or device write-only
> +	part of the message, is too small.
> +\end{itemize}
> +
> +VIRTIO_RTC_S_EIO indicates that the device did not execute the request
> +due to an error which was not caused by invalid input from the driver.
> +
> +All \field{reserved} fields are written as zero, and their value is
> +ignored.
> +
> +The set of clocks does not change after feature negotiation completion,
> +until device reset. The set of clocks should not change on device reset
> +either (similar to negotiated features). Clock identifiers are
> +zero-based, dense indices. All fields named \field{clock_id} contain
> +clock identifiers.
> +
> +\subsubsection{Control Requests}\label{sec:Device Types / RTC Device /
> +Device Operation / Control Requests}
> +
> +Through \emph{control requests}, the driver requests information about
> +the device capabilities. The driver enqueues control requests in the
> +requestq.
> +
> +\begin{description}
> +
> +\item[VIRTIO_RTC_REQ_CFG] discovers the number of clocks.
> +
> +\begin{lstlisting}
> +#define VIRTIO_RTC_REQ_CFG 0x1000 /* message type */
> +
> +struct virtio_rtc_req_cfg {
> +	struct virtio_rtc_req_head head;
> +	/* no request params */
> +};
> +
> +struct virtio_rtc_resp_cfg {
> +	struct virtio_rtc_resp_head head;
> +	le16 num_clocks;
> +	u8 reserved[6];
> +};
> +\end{lstlisting}
> +
> +Field \field{num_clocks} contains the number of clocks. A device
> +provides zero or more clocks. Valid clock ids are those smaller than
> +\field{num_clocks}.
> +
> +\item[VIRTIO_RTC_REQ_CLOCK_CAP] discovers the capabilities of the clock
> +identified by the \field{clock_id} field.
> +
> +\begin{lstlisting}
> +#define VIRTIO_RTC_REQ_CLOCK_CAP 0x1001 /* message type */
> +
Each clock identified by unique resource (clock) identifier and this resource now has the attributes of time.

> +struct virtio_rtc_req_clock_cap {
> +	struct virtio_rtc_req_head head;
> +	le16 clock_id;
> +	u8 reserved[6];
> +};
> +
> +struct virtio_rtc_resp_clock_cap {
> +	struct virtio_rtc_resp_head head;
> +	le16 type;
> +	u8 reserved[6];
> +};
> +\end{lstlisting}
> +
> +The \field{type} field identifies the clock type. A device provides
> +zero or more clocks for a clock type. The following clock types are
> +defined:
> +
> +\begin{lstlisting}
> +#define VIRTIO_RTC_CLOCK_UTC 0
> +#define VIRTIO_RTC_CLOCK_TAI 1
> +#define VIRTIO_RTC_CLOCK_MONO 2
> +\end{lstlisting}
> +
> +VIRTIO_RTC_CLOCK_UTC uses the UTC (Coordinated Universal Time) time
> +standard. The device uses the time epoch of January 1, 1970, 00:00 UTC.
> +This is the same epoch as \emph{Unix time}.
> +
> +VIRTIO_RTC_CLOCK_TAI uses the TAI (International Atomic Time) time
> +standard. The device uses the time epoch of January 1, 1970, 00:00 TAI.
> +
> +VIRTIO_RTC_CLOCK_MONO uses monotonic physical time (SI seconds
> +subdivisions) since some unspecified epoch. The
> VIRTIO_RTC_CLOCK_MONO
> +epoch is before or during device reset.
> +
> +Additional clock types may be standardized in the future.
> +Implementation-specific clock type definitions are not recommended and
> +use a clock type id greater than or equal to 0xF000.
> +
> +\item[VIRTIO_RTC_REQ_CROSS_CAP] discovers whether the device
> supports
> +cross-timestamping for a particular pair of clock and hardware counter.
> +
> +\begin{lstlisting}
> +#define VIRTIO_RTC_REQ_CROSS_CAP 0x1002 /* message type */
> +
> +struct virtio_rtc_req_cross_cap {
> +	struct virtio_rtc_req_head head;
> +	le16 clock_id;
> +	le16 hw_counter;
> +	u8 reserved[4];
> +};
> +
> +#define VIRTIO_RTC_FLAG_CROSS_CAP (1 << 0)
> +
> +struct virtio_rtc_resp_cross_cap {
> +	struct virtio_rtc_resp_head head;
> +	u8 flags;
> +	u8 reserved[7];
> +};
> +\end{lstlisting}
> +
> +The \field{clock_id} field identifies the clock, and the
> +\field{hw_counter} field identifies the hardware counter, for which
> +cross-timestamp support is probed. The device sets flag
> +\field{VIRTIO_RTC_FLAG_CROSS_CAP} in the \field{flags} field if the
> +clock supports cross-timestamping for the particular clock and hardware
> +counter, and clears the flag otherwise.
> +
Hw_counter is also the resource attribute.

> +\end{description}
> +
> +\subsubsection{Read Requests}\label{sec:Device Types / RTC Device /
> +Device Operation / Read Requests}
> +
> +Through \emph{read requests}, the driver requests clock readings from
> +the device. The driver enqueues read requests in the requestq. The
> +device obtains device-side clock readings and forwards these clock
> +readings to the driver. The driver may enhance and interpret the clock
> +readings through methods which are beyond the scope of this
> +specification.
> +
Reading the resource using generic query resource command of [2] will return the time value.
So only time definition is needed in the resource.
[2] https://lists.oasis-open.org/archives/virtio-comment/202401/msg00096.html

> +Once DRIVER_OK has been set, the device should support reading every
> +clock, even when a clock may yet have to be aligned to reference time
> +sources.
> +
> +In general,
> +
> +\begin{itemize}
> +\item clocks may jump backwards or forward, and \item the clock
> +frequency may change. Clocks may be \emph{slewed},
> +	i.e.\ clocks may run at a frequency other than their current
> +	best frequency estimate.
> +\end{itemize}
> +
> +E.g., a clock of type VIRTIO_RTC_CLOCK_UTC can insert UTC leap seconds.
> +
> +As long as a clock does not jump backwards, the driver clock readings
> +increase monotonically:
> +
> +\begin{itemize}
> +\item As long as a clock does not jump backwards in-between device-side
> +	clock readings, the driver-side readings for that clock increase
> +	monotonically as well, in the order in which the driver
> +	marks read requests as available.
> +
> +\item The device marks read requests for the same clock as used in
> +	the order in which the messages were marked as available.
> +\end{itemize}
> +
> +For a clock of type VIRTIO_RTC_CLOCK_MONO, the device always returns
> +monotonically increasing clock readings.
> +
> +The unit of all \field{clock_reading} fields is 1 nanosecond for all
> +clock types which are part of the device specification so
> +far.\footnote{For time epochs in year 1970 or later, this means that
> +time until at least year 2553 can be represented in the \field{le64
> +clock_reading} fields.}
> +
> +\begin{description}
> +
> +\item[VIRTIO_RTC_REQ_READ] reads the clock identified by the
> +\field{clock_id} field. The device supports this message for every
> +clock.
> +
> +\begin{lstlisting}
> +#define VIRTIO_RTC_REQ_READ 0x0001 /* message type */
> +
> +struct virtio_rtc_req_read {
> +	struct virtio_rtc_req_head head;
> +	le16 clock_id;
> +	u8 reserved[6];
> +};
> +
> +struct virtio_rtc_resp_read {
> +	struct virtio_rtc_resp_head head;
> +	le64 clock_reading;
> +};
> +\end{lstlisting}
> +
> +\field{clock_reading} is a device-side clock reading obtained after the
> +message was marked as available.
> +
> +\item[VIRTIO_RTC_REQ_READ_CROSS] returns a cross-timestamp for the
> +clock identified by the \field{clock_id}
> +field.\footnote{Cross-timestamping
> +is similar to the ptp_kvm mechanism in the Linux kernel.} This message
> +may yield better performance than using VIRTIO_RTC_REQ_READ.
> +
> +\begin{lstlisting}
> +#define VIRTIO_RTC_REQ_READ_CROSS 0x0002 /* message type */
> +
> +struct virtio_rtc_req_read_cross {
> +	struct virtio_rtc_req_head head;
> +	le16 clock_id;
> +	le16 hw_counter;
> +	u8 reserved[4];
> +};
> +
> +struct virtio_rtc_resp_read_cross {
> +	struct virtio_rtc_resp_head head;
> +	le64 clock_reading;
> +	le64 counter_cycles;
> +};
> +\end{lstlisting}
> +
> +The \field{hw_counter} field specifies the hardware counter for which
> +the driver requests a cross-timestamp.
> +
> +Cross-timestamping returns a \field{clock_reading}, and an associated
> +hardware counter value, \field{counter_cycles}. The
> +\field{counter_cycles} field is the approximate or precise value which
> +the driver would have read at the \field{clock_reading} time instant
> +from the hardware counter identified by \field{hw_counter}. How the
> +device determines the value which the driver would have seen is beyond
> +the scope of this specification.
> +
> +The following hardware counter identifiers are specified:
> +
> +\begin{lstlisting}
> +/* Arm Generic Timer CNTVCT_EL0, Counter-timer Virtual Count register
> +*/ #define VIRTIO_RTC_COUNTER_ARM_VIRT 0
> +/* Arm Generic Timer CNTPCT_EL0, Counter-timer Physical Count register
> +*/ #define VIRTIO_RTC_COUNTER_ARM_PHYS 1
> +/* x86 Time-Stamp Counter */
> +#define VIRTIO_RTC_COUNTER_X86_TSC 2
> +\end{lstlisting}
> +
> +Additional hardware counter identifiers may be standardized in the
> +future. Implementation-specific hardware counter identifiers are not
> +recommended and are greater than or equal to 0xF000.
> +
> +The driver can determine whether the device supports
> +VIRTIO_RTC_REQ_READ_CROSS for a specific clock and \field{hw_counter}
> +through VIRTIO_RTC_REQ_CROSS_CAP.
> +
> +\end{description}
> --
> 2.40.1