OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

virtio-comment message

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]


Subject: Re: [virtio-comment] [PATCH] virtio-net: Add support for the flexible driver notification


>-----Original Message-----
>From: virtio-comment@lists.oasis-open.org <virtio-comment@lists.oasis-
>open.org> On Behalf Of Michael S. Tsirkin
>Sent: Monday, 27 January, 2020 20:48
>To: Vitaly Mireyno <vmireyno@marvell.com>
>Cc: Stefan Hajnoczi <stefanha@redhat.com>; virtio-comment@lists.oasis-
>open.org; Jason Wang <jasowang@redhat.com>
>Subject: [EXT] Re: [virtio-comment] [PATCH] virtio-net: Add support for the
>flexible driver notification
>
>On Mon, Jan 27, 2020 at 05:00:03PM +0000, Vitaly Mireyno wrote:
>>
>> >-----Original Message-----
>> >From: Michael S. Tsirkin <mst@redhat.com>
>> >Sent: Monday, 27 January, 2020 17:34
>> >To: Vitaly Mireyno <vmireyno@marvell.com>
>> >Cc: Stefan Hajnoczi <stefanha@redhat.com>;
>> >virtio-comment@lists.oasis- open.org; Jason Wang
>> ><jasowang@redhat.com>
>> >Subject: [EXT] Re: [virtio-comment] [PATCH] virtio-net: Add support
>> >for the flexible driver notification
>> >
>> >On Mon, Jan 27, 2020 at 12:49:48PM +0000, Vitaly Mireyno wrote:
>> >>
>> >> >-----Original Message-----
>> >> >From: Stefan Hajnoczi <stefanha@redhat.com>
>> >> >Sent: Monday, 27 January, 2020 11:50
>> >> >To: Vitaly Mireyno <vmireyno@marvell.com>
>> >> >Cc: virtio-comment@lists.oasis-open.org; Michael S. Tsirkin
>> >> ><mst@redhat.com>; Jason Wang <jasowang@redhat.com>
>> >> >Subject: [EXT] Re: [virtio-comment] [PATCH] virtio-net: Add
>> >> >support for the flexible driver notification
>> >> >
>> >> >On Sun, Jan 26, 2020 at 12:10:28PM +0000, Vitaly Mireyno wrote:
>> >> >> Currently, the driver notification (available buffer
>> >> >> notification) has a fixed
>> >> >structure.
>> >> >> If VIRTIO_F_NOTIFICATION_DATA has been negotiated, it includes:
>> >> >> vqn,
>> >> >next_off and next_wrap.
>> >> >> If notify_off_multiplier > 0, the VQ number can be derived by
>> >> >> the device
>> >> >from the Queue Notify address, so vqn may be redundant.
>> >> >>
>> >> >> Some devices benefit from receiving an additional data with
>> >> >> driver
>> >> >notifications (because of the specific hardware implementation).
>> >> >This data can optionally replace the vqn field in the driver notification
>structure.
>> >> >> In its simplest form, it would be sufficient for this data to be
>> >> >> a per-device
>> >> >constant value.
>> >> >>
>> >> >>
>> >> >> Signed-off-by: Vitaly Mireyno <vmireyno@marvell.com>
>> >> >> ---
>> >> >>  content.tex | 24 ++++++++++++++++++++++++
>> >> >>  1 file changed, 24 insertions(+)
>> >> >
>> >> >Hi Vitaly,
>> >> >Can you give a concrete example of how devices can use this feature?
>> >> >
>> >> >The reason I'm a little unsure is that the PCI capability only
>> >> >provides a single notify_data value per device.  That same value
>> >> >is then passed back to the device in the notify write.
>> >> >
>> >> >If there is just a single constant value per device, why is it
>> >> >necessary to involve the driver at all?  Doesn't the device
>> >> >already know its own magic value and it doesn't really need the
>> >> >driver to pass the
>> >value back to it?
>> >> >
>> >> >Stefan
>> >> >
>> >>
>> >> This will help HW devices that share the same memory space between
>> >virtqueue notifications and other types of notifications. The virtio
>> >driver will always use the value that will tell the HW that it's a virtqueue
>notification.
>> >> Of course, the feature can be extended, such that there will be a
>> >> value
>> >configurable per virtqueue (which is a generalization of the vqn
>> >field). I just didn't want to complicate it further, without a real-life
>justification.
>> >
>> >I would like to see a bit more detail about these other types of
>> >notifications then.
>> >
>>
>> Other notification types could be control-plane notifications, for example
>adding MAC/VLAN filter. These types of notifications will not be sent by the
>virtio driver, but rather by a vendor driver in the vDPA framework.
>> Another example is offloaded RDMA notifications.
>> The thing is that the HW doesn't have a "mode" configuration that can
>distinguish between virtio and other modes. The HW only reacts to the
>notification data, which is expected to include all the relevant information.
>
>If it's control plane, isn't the natural way to do this is by mapping a separate
>range of addresses though?
>
>It might be a good idea to add extra data in the notifications but wasting these
>bits on control plane things seems like a weird design choice.
>

You are right, but the control plane is only an example. It's mainly designed to be used for other offloaded protocols data plane notifications, and since the HW infrastructure is already there, the control plane may as well use it.
Sharing the same address range helps reducing the PCI BAR size.

>
>> >
>> >>
>> >> >> diff --git a/content.tex b/content.tex index 06bb4ca..265dc12
>> >> >> 100644
>> >> >> --- a/content.tex
>> >> >> +++ b/content.tex
>> >> >> @@ -965,6 +965,9 @@ \subsubsection{Notification structure
>> >> >> layout}\label{sec:Virtio Transport Options  struct virtio_pci_notify_cap
>{
>> >> >>          struct virtio_pci_cap cap;
>> >> >>          le32 notify_off_multiplier; /* Multiplier for
>> >> >> queue_notify_off. */
>> >> >> +        le16 notify_data; /* The data to be placed in the vqn field */
>> >> >> +        u8 notify_data_valid;
>> >> >> +        u8 padding; /* Pad to a dword */
>> >> >>  };
>> >> >>  \end{lstlisting}
>> >> >>
>> >> >> @@ -984,6 +987,9 @@ \subsubsection{Notification structure
>> >> >> layout}\label{sec:Virtio Transport Options  the same Queue
>> >> >> Notify address
>> >> >for all queues.
>> >> >>  \end{note}
>> >> >>
>> >> >> +If \field{notify_data_valid} != 0, the driver will set the
>> >> >> +\field{vqn} field in the available buffer notification
>> >> >> +structure to the
>> >> >\field{notify_data} value.
>> >> >> +
>> >> >>  \devicenormative{\paragraph}{Notification capability}{Virtio
>> >> >> Transport Options / Virtio Over PCI Bus / PCI Device Layout /
>> >> >> Notification
>> >> >capability}  The device MUST present at least one notification capability.
>> >> >>
>> >> >> @@ -1020,6 +1026,12 @@ \subsubsection{Notification structure
>> >> >> layout}\label{sec:Virtio Transport Options  cap.length >=
>> >> >> queue_notify_off * notify_off_multiplier + 4  \end{lstlisting}
>> >> >>
>> >> >> +The device MAY set \field{notify_data_valid} to any non-zero
>> >> >> +value, to request the driver to set the \field{vqn} to the
>> >\field{notify_data} value.
>> >> >> +If the device sets \field{notify_data_valid} to a non-zero
>> >> >> +value, it MUST set \field{notify_data} to a valid value.
>> >> >> +For a normal operation, the device MUST set
>> >> >> +\field{notify_data_valid} to
>> >> >zero.
>> >> >> +
>> >> >>  \subsubsection{ISR status capability}\label{sec:Virtio
>> >> >> Transport Options / Virtio Over PCI Bus / PCI Device Layout /
>> >> >> ISR status capability}
>> >> >>
>> >> >>  The VIRTIO_PCI_CAP_ISR_CFG capability @@ -1519,6 +1531,18 @@
>> >> >> \subsubsection{Available Buffer Notifications}\label{sec:Virtio
>> >> >> Transport Option  See \ref{sec:Virtio Transport Options / Virtio
>> >> >> Over PCI Bus / PCI Device Layout / Notification capability}  for
>> >> >> how to calculate the
>> >> >Queue Notify address.
>> >> >>
>> >> >> +\drivernormative{\paragraph}{Available Buffer
>> >> >> +Notifications}{Virtio Transport Options / Virtio Over PCI Bus /
>> >> >> +PCI-specific Initialization And Device Operation / Available
>> >> >> +Buffer Notifications} If VIRTIO_F_NOTIFICATION_DATA has been
>> >> >> +negotiated, and if
>> >> >\field{notify_data_valid} != 0, the driver MUST set the
>> >> >\field{vqn} field of the available buffer notification structure
>> >> >to the \field{notify_data}
>> >value.
>> >> >> +
>> >> >> +\begin{note}
>> >> >> +If \field{notify_off_multiplier} > 0, the virtqueue number can
>> >> >> +potentially be derived by the device from the Queue Notify
>> >> >> +address, so \field{vqn} may be redundant. Some devices benefit
>> >> >> +from receiving the additional data with driver notifications
>> >> >> +(because of the specific
>> >> >hardware implementation).
>> >> >> +\end{note}
>> >> >> +
>> >> >>  \subsubsection{Used Buffer Notifications}\label{sec:Virtio
>> >> >> Transport Options / Virtio Over PCI Bus / PCI-specific
>> >> >> Initialization And Device Operation / Used Buffer Notifications}
>> >> >>
>> >> >>  If a used buffer notification is necessary for a virtqueue, the
>> >> >> device would
>> >> >typically act as follows:
>> >> >> --
>> >> >>
>> >> >> This publicly archived list offers a means to provide input to
>> >> >> the OASIS Virtual I/O Device (VIRTIO) TC.
>> >> >>
>> >> >> In order to verify user consent to the Feedback License terms
>> >> >> and to minimize spam in the list archive, subscription is
>> >> >> required before posting.
>> >> >>
>> >> >> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
>> >> >> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
>> >> >> List help: virtio-comment-help@lists.oasis-open.org
>> >> >> List archive:
>> >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.oasis
>> >> >> -2D
>> >> >> open.org_archives_virtio-
>> >2Dcomment_&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xt
>> >> >>
>>
>>fQ&r=lDHJ2FW52oJ3lqqsArgFRdcevq01tbLQAw4A_NO7xgI&m=V_WFWAet
>W
>> >bHu0u2N
>> >> >>
>>
>>hwNHEqFKyE_y6sUMeJcHqURDeoA&s=xTzOJ9VTGfxRUMJf0A4unV7kB_hd
>Ml
>> >oGKMKrQ
>> >> >> hnSJKc&e=
>> >> >> Feedback License:
>> >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oasis-
>> >2Dop
>> >> >> en.org_who_ipr_feedback-
>> >5Flicense.pdf&d=DwIFAg&c=nKjWec2b6R0mOyPaz7
>> >> >>
>>
>>xtfQ&r=lDHJ2FW52oJ3lqqsArgFRdcevq01tbLQAw4A_NO7xgI&m=V_WFWAe
>t
>> >WbHu0u
>> >> >>
>>
>>2NhwNHEqFKyE_y6sUMeJcHqURDeoA&s=vL3tJ_XMSNJwV8gDnLft9liDG6oS
>V
>> >HLNhdd
>> >> >> MUe4ZZNI&e=
>> >> >> List Guidelines:
>> >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oasis-
>> >2Dop
>> >> >> en.org_policies-2Dguidelines_mailing-
>> >2Dlists&d=DwIFAg&c=nKjWec2b6R0
>> >> >>
>>
>>mOyPaz7xtfQ&r=lDHJ2FW52oJ3lqqsArgFRdcevq01tbLQAw4A_NO7xgI&m=V
>_
>> >WFWAe
>> >> >>
>>
>>tWbHu0u2NhwNHEqFKyE_y6sUMeJcHqURDeoA&s=A444Hd_Yb4aycXM5Ids
>B
>> >41E4xDmF
>> >> >> M7c3G7m8tQZCvBg&e=
>> >> >> Committee:
>> >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oasis-
>> >2Dop
>> >> >>
>>
>>en.org_committees_virtio_&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=lD
>> >HJ2
>> >> >>
>>
>>FW52oJ3lqqsArgFRdcevq01tbLQAw4A_NO7xgI&m=V_WFWAetWbHu0u2Nh
>w
>> >NHEqFKyE
>> >> >>
>>
>>_y6sUMeJcHqURDeoA&s=VA420pM5tOIIqUWH7lCMCkm2zbEbtBBOvotp9g
>W
>> >r9QM&e=
>> >> >> Join OASIS:
>> >> >> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.oasis-
>> >2Dop
>> >> >>
>>
>>en.org_join_&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=lDHJ2FW52oJ3lq
>q
>> >sAr
>> >> >>
>>
>>gFRdcevq01tbLQAw4A_NO7xgI&m=V_WFWAetWbHu0u2NhwNHEqFKyE_y6
>s
>> >UMeJcHqUR
>> >> >> DeoA&s=hVpH-MUm5o9XpvEa6BOWraE8-45dGcCFvBhGRITH_nE&e=
>> >> >>
>
>
>This publicly archived list offers a means to provide input to the OASIS Virtual
>I/O Device (VIRTIO) TC.
>
>In order to verify user consent to the Feedback License terms and to minimize
>spam in the list archive, subscription is required before posting.
>
>Subscribe: virtio-comment-subscribe@lists.oasis-open.org
>Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
>List help: virtio-comment-help@lists.oasis-open.org
>List archive: https://urldefense.proofpoint.com/v2/url?u=https-
>3A__lists.oasis-2Dopen.org_archives_virtio-
>2Dcomment_&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=lDHJ2FW52oJ3lq
>qsArgFRdcevq01tbLQAw4A_NO7xgI&m=ZwtynokTxS59JQ6NCjDu8EpKsv1ZL-
>7pzZ7LKd4663I&s=6ej9qQAg0jeBVJNhzfumR3Q7eLhFNNWLpz5Zlk9KIVM&e=
>Feedback License: https://urldefense.proofpoint.com/v2/url?u=https-
>3A__www.oasis-2Dopen.org_who_ipr_feedback-
>5Flicense.pdf&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=lDHJ2FW52oJ3lqq
>sArgFRdcevq01tbLQAw4A_NO7xgI&m=ZwtynokTxS59JQ6NCjDu8EpKsv1ZL-
>7pzZ7LKd4663I&s=ivzoy5NStwJg3_x8GzB-XNSOykSYt1Sy7Yr9w4JZ7ts&e=
>List Guidelines: https://urldefense.proofpoint.com/v2/url?u=https-
>3A__www.oasis-2Dopen.org_policies-2Dguidelines_mailing-
>2Dlists&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=lDHJ2FW52oJ3lqqsArgFR
>dcevq01tbLQAw4A_NO7xgI&m=ZwtynokTxS59JQ6NCjDu8EpKsv1ZL-
>7pzZ7LKd4663I&s=Qi0rg76WzE90Oienx5UWxgIAWu27yejzprteF-WguqM&e=
>Committee: https://urldefense.proofpoint.com/v2/url?u=https-
>3A__www.oasis-
>2Dopen.org_committees_virtio_&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xtfQ&
>r=lDHJ2FW52oJ3lqqsArgFRdcevq01tbLQAw4A_NO7xgI&m=ZwtynokTxS59JQ6
>NCjDu8EpKsv1ZL-
>7pzZ7LKd4663I&s=N9LxnUJBE6Ev1UGHjD8vbFhIWML8NF5PnVVS7N2KYSE&e=
>Join OASIS: https://urldefense.proofpoint.com/v2/url?u=https-
>3A__www.oasis-
>2Dopen.org_join_&d=DwIFAg&c=nKjWec2b6R0mOyPaz7xtfQ&r=lDHJ2FW52o
>J3lqqsArgFRdcevq01tbLQAw4A_NO7xgI&m=ZwtynokTxS59JQ6NCjDu8EpKsv1
>ZL-
>7pzZ7LKd4663I&s=1euFBF86OgTr6gTGn0dUY8oY8bh6DqUoWGX4ag2Kcs0&e=



[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]