[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-comment] About the plan of Admin Queue
On Mon, 3 Jul 2023 16:01:35 +0800, "Zhu, Lingshan" <lingshan.zhu@intel.com> wrote: > > > On 7/3/2023 1:54 PM, Xuan Zhuo wrote: > > On Mon, 3 Jul 2023 12:29:32 +0800, "Zhu, Lingshan" <lingshan.zhu@intel.com> wrote: > >> > >> On 6/30/2023 7:35 PM, Parav Pandit wrote: > >>>> From: virtio-comment@lists.oasis-open.org <virtio-comment@lists.oasis- > >>>> open.org> On Behalf Of Zhu, Lingshan > >>>> Sent: Friday, June 30, 2023 6:33 AM > >>>> > >>>>>>> Can we let the DPU notify the driver to create a new devicer from the > >>>> backend? > >>> Yes, why not. > >>> > >>>>>>> The key point is who want to create a new device. > >>>>>> DPU can come with a certain number of pre-created ADIs, just make > >>>>>> sure the orchestration SW is aware of their device IDs. > >>>>>> > >>> Cloud often need these devices to be created dynamically, many a time after the host OS is booted. > >>> To be more generic, those devices to be created and connected to the host regardless of the life cycle of the host. > >>> Xuan partly explained it. > >>> > >>>>>> If you want the DPU randomly create ADIs and notify the driver, I > >>>>>> think we need interrupt, e.g., re-use config interrupt. But why DPU > >>>>>> wants to create and hot plug in a device to a guest? > >>>>>> Shall the host handle that or DPU pre-create then expose to baremteal > >>>>>> machines? > >>>>> In your scenario, the supervisor is on the os, which controls the DPU > >>>>> to create new devices. > >>>>> > >>>>> In the cloud scenario, the vendor manager is in the DPU, and the > >>>>> entire host is for users. Of course, there are situations where the > >>>>> vendor manager are in the HOST. But for bare metal machines, the host > >>>>> belongs to the customer, the vendor manager is only in the DPU. > >>>>> > >>>>> So when the customers buy a new nic for the host, the vendor manager > >>>>> will plug a device to the host from the DPU. > >>>> I understand once a customer orders a new NIC, you wants to present the NIC > >>>> to the host. > >>>> However you only owns the DPU and the customer owns the host, that means > >>>> this creation and hot plug must be transparent to the host and there may not be > >>>> a host driver help handling an interrupt/probe. > >>>> > >>> That is ok. when driver is loaded, it would query about its child devices and probe it, if we strictly want to follow SIOV model. > >>> > >>>> However this is not PCI which has a tree/switch and can enumerate devices to > >>>> the host by spanning the device across the PCI hierarchy. > >>>> > >>> Those enumeration is triggered by the parent PCI device and pci bridge and switch will also discover it. > >>> > >>>> To address an ADI, there is only a device_id. > >>>> > >>> SIOV device must have a unique identifier at PCI bus level for sure. > >>> I cannot speak more about it in this forum due to other logistics issue. > >>> But assume that there is PCI level unique identifier for SIOV device that switches on the path will learn about. > >>> > >>>> So, do you mind share how your DPU offload the device model? What kind of > >>>> device your DPU provide to the host? Lets see whether DPU can mediate this by > >>>> its own? > >>>> > >>> It is a virtio nic, blk and other virtio devices for us. > >>> A DPU hotplugs a device, host side either gets interrupt or later gets to know about it when explicitly queries. > >>> There is no mediation per say here, it is just a dpu based SIOV device like a regular PF. > >>> > >>> For non virtio DPU device, I implemented them in Linux for dpus 2 years ago. > >>> You might find a Linux reference model useful at [1]. > >>> A usage model already exists in one OS and in use for non virtio devices. > >>> This certainly works without SIOV unique PCI device identifiers, because DPU (non-host) managed SIOV device spec still does not exist. > >>> > >>> For virtio, I think we should wait for this piece to be defined and leverage that, instead of virtio tc creating its own. > >>> > >>> [1] https://github.com/Mellanox/scalablefunctions/wiki > >> well I see SF facing the similar challenge, I can add a command for the > >> driver to query all existing SIOV ADIs of a device, > >> and the device return ADIs id and status. Looks good? and work for you > >> @Xuan? > > Can the device notifies the driver by a interrupt? > Of course this can be an option as previously discussed. > That would be a config interrupt(with proper suppression) from the > management device, > which requires a new cap in a bar, then you still need to fetch the > information. > > A proposal: > > a new cap, at least contains: > total number of the ADIs. > > Then once any ADIs are created or removed, the PF driver get a notification. > > Works for you? Look good to me. Thanks. > > > > Thanks. > > > > > >> Thanks > >> > > This publicly archived list offers a means to provide input to the > > OASIS Virtual I/O Device (VIRTIO) TC. > > > > In order to verify user consent to the Feedback License terms and > > to minimize spam in the list archive, subscription is required > > before posting. > > > > Subscribe: virtio-comment-subscribe@lists.oasis-open.org > > Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org > > List help: virtio-comment-help@lists.oasis-open.org > > List archive: https://lists.oasis-open.org/archives/virtio-comment/ > > Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf > > List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists > > Committee: https://www.oasis-open.org/committees/virtio/ > > Join OASIS: https://www.oasis-open.org/join/ > > >
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]