Re: [PATCH v10 03/10] admin: introduce group administration commands

[Stefan Hajnoczi <stefanha@redhat.com>]

On Thu, Mar 02, 2023 at 06:57:24PM -0500, Michael S. Tsirkin wrote:
> On Thu, Mar 02, 2023 at 03:10:11PM -0500, Stefan Hajnoczi wrote:
> > On Thu, Mar 02, 2023 at 08:05:02AM -0500, Michael S. Tsirkin wrote:
> > > This introduces a general structure for group administration commands,
> > > used to control device groups through their owner.
> > > 
> > > Following patches will introduce specific commands and an interface for
> > > submitting these commands to the owner.
> > > 
> > > Signed-off-by: Max Gurtovoy <mgurtovoy@nvidia.com>
> > > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> > > ---
> > >  admin.tex        | 108 +++++++++++++++++++++++++++++++++++++++++++++++
> > >  introduction.tex |   3 ++
> > >  2 files changed, 111 insertions(+)
> > > 
> > > diff --git a/admin.tex b/admin.tex
> > > index 3dc47be..7e28b77 100644
> > > --- a/admin.tex
> > > +++ b/admin.tex
> > > @@ -46,4 +46,112 @@ \section{Device groups}\label{sec:Basic Facilities of a Virtio Device / Device g
> > >  PCI transport (see \ref{sec:Virtio Transport Options / Virtio Over PCI Bus}).
> > >  \end{description}
> > >  
> > > +\subsection{Group administration commands}\label{sec:Basic Facilities of a Virtio Device / Device groups / Group administration commands}
> > >  
> > > +The driver sends group administration commands to the owner device of
> > 
> > I notice that the terminology is simply "the driver". "Owner driver"
> > and "group member driver" might be clearer because there will be two
> > (possibly different) drivers involved.
> 
> Hmm I don't really want to repeat owner everywhere.
> I will clarify that in this section simple "driver" and "device" are
> owner, "member device" and "member driver" is always called explicitly.

Sounds good.

> > > +a group to control member devices of the group.
> > > +This mechanism can
> > > +be used, for example, to configure a member device before it is
> > > +initialized by its driver.
> > > +\footnote{The term "administration" is intended to be interpreted
> > > +widely to include any kind of control. See specific commands
> > > +for detail.}
> > > +
> > > +All the group administration commands are of the following form:
> > > +
> > > +\begin{lstlisting}
> > > +struct virtio_admin_cmd {
> > > +        /* Device-readable part */
> > > +        le16 opcode;
> > > +        /*
> > > +         * 1 - SR-IOV
> > > +         * 2 - 65535 reserved
> > > +         */
> > > +        le16 group_type;
> > > +        /* unused, reserved for future extensions */
> > > +        u8 reserved1[12];
> > > +        le64 group_member_id;
> > > +        u8 command_specific_data[];
> > > +
> > > +        /* Device-writable part */
> > > +        le16 status;
> > > +        le16 status_qualifier;
> > > +        /* unused, reserved for future extensions */
> > > +        u8 reserved2[4];
> > > +        u8 command_specific_result[];
> > > +};
> > > +\end{lstlisting}
> > > +
> > > +For all commands, \field{opcode}, \field{group_type} and if
> > > +necessary \field{group_member_id} and \field{command_specific_data} are
> > > +set by the driver, and the owner device sets \field{status} and if
> > > +needed \field{status_qualifier} and
> > > +\field{command_specific_result}.
> > > +
> > > +Generally, any unused device-readable fields are set to zero by the driver
> > > +and ignored by the device.  Any unused device-writeable fields are set to zero
> > > +by the device and ignored by the driver.
> > > +
> > > +\field{opcode} specifies the command. The valid
> > > +values for \field{opcode} can be found in the following table:
> > > +
> > > +\begin{tabular}{|l|l|}
> > > +\hline
> > > +opcode & Name & Command Description \\
> > > +\hline \hline
> > > +0x0000 - 0x7FFF & - &  Group administration commands    \\
> > > +\hline
> > > +0x8000 - 0xFFFF & - & Reserved    \\
> > > +\hline
> > > +\end{tabular}
> > 
> > I thought all commands are "group administration commands" but this
> > table makes it look like they are just a subset (0x0000 - 0x7FFF) of
> > group administration commands, which is a paradox.
> 
> Well the rest are reserved, maybe we will have more command types who
> knows. No?

I see. Does that mean the reserved commands don't need to be in the same
format as struct virtio_admin_cmd?

The entire section is called "Group administration commands" but I get
the impression it's talking both about admin virtqueue commands in
general and specifically about group administration commands.

Is it possible to structure this as follows:

  Admin Commands
    ...common stuff...
    Group Administration Commands (0x0000-0x7fff)
      ...specific to group administration commands...
    Reserved (0x8000-0xffff)

> > > +
> > > +The \field{group_type} specifies the group type identifier.
> > > +The \field{group_member_id} specifies the member identifier within the group.
> > > +See section \ref{sec:Introduction / Terminology / Device group}
> > > +for the definition of the group type identifier and group member
> > > +identifier.
> > > +
> > > +The following table describes possible \field{status} values;
> > > +to simplify common implementations, they are intentionally
> > > +matching common \hyperref[intro:errno]{Linux error names and numbers}:
> > > +
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Status (decimal) & Name & Description \\
> > > +\hline \hline
> > > +00   & VIRTIO_ADMIN_STATUS_OK    & successful completion  \\
> > > +\hline
> > > +22   & VIRTIO_ADMIN_STATUS_EINVAL    & invalid command \\
> > > +\hline
> > > +other   & -    & group administration command error  \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +When \field{status} is VIRTIO_ADMIN_STATUS_OK, \field{status_qialifier}
> > 
> > s/qialifier/qualifier/
> > 
> > > +is reserved and set to zero by the device.
> > > +
> > > +When \field{status} is VIRTIO_ADMIN_STATUS_EINVAL,
> > > +the following table describes possible \field{status_qialifier} values:
> > 
> > s/qialifier/qualifier/
> > 
> > > +\begin{tabular}{|l|l|l|}
> > > +\hline
> > > +Status & Name & Description \\
> > > +\hline \hline
> > > +0x00   & VIRTIO_ADMIN_STATUS_Q_INVALID_COMMAND   & command error: no additional information  \\
> > > +\hline
> > > +0x01   & VIRTIO_ADMIN_STATUS_Q_INVALID_OPCODE    & unsupported or invalid \field{opcode}  \\
> > > +\hline
> > > +0x02   & VIRTIO_ADMIN_STATUS_Q_INVALID_FIELD    & unsupported or invalid field within \field{command_specific_data}  \\
> > > +\hline
> > > +0x03   & VIRTIO_ADMIN_STATUS_Q_INVALID_GROUP    & unsupported or invalid \field{group_type} \\
> > > +\hline
> > > +0x04   & VIRTIO_ADMIN_STATUS_Q_INVALID_MEMBER    & unsupported or invalid \field{group_member_id} \\
> > > +\hline
> > > +0x05-0xFFFF   & -    & reserved for future use \\
> > > +\hline
> > > +\end{tabular}
> > > +
> > > +Each command uses a different \field{command_specific_data} and
> > > +\field{command_specific_result} structures and the length of
> > > +\field{command_specific_data} and \field{command_specific_result}
> > > +depends on these structures and is described separately or is
> > > +implicit in the structure description.
> > 
> > On more thing:
> > 
> > Does the owner device see commands in order but may complete them in any
> > order?
> > 
> > I think this information might be useful just to make it clear that
> > driver authors shouldn't make assumptions about ordering and completion
> > order, e.g. pipelining a bunch of dependent commands doesn't work
> > because the first command is not necessarily completed before the second
> > command is started.
> 
> I think this is discussed when we discuss sending commands through vq.
> in this section it's just about commands generally whatever the way
> to send them to device. no?

Okay. I'm reading this patch series linearly and the question popped
into my mind here. Glad it's already covered elsewhere.

Stefan

Attachment: signature.asc
Description: PGP signature