[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: Re: [virtio-comment] About adding a new device type virtio-nvme
On Thu, 19 Jan 2023 11:38:51 +0800ïJason Wang wrote:
>å 2023/1/18 10:15, äèä åé:
>> On Tue, 17 Jan 2023 10:34:09 -0500, Stefan wrote:
>>
>> On Tue, Jan 17, 2023 at 05:41:57PM +0800, äèä wrote:
>>>> On Tue, 17 Jan 2023 09:32:05 +0100ïDavid wroteï
>>>> On 17.01.23 03:04, äèä wrote:
>>>>
>>>>
>>>>>> virtio-nvmeÂadvantages :
>>>>>> 1)Â live migration
>>>>>> 2)Â support remote storage
>>>>
>>>>
>>>>> At least 1) is an implementation detail in the NVME implementation in
>>>> the hypervisor. I suspect 2) in a similar way, or is there a fundamental
>>>>> issue with that?
>>>>
>>>>
>>>>> One problematic thing about the NVME implementation in QEMU is that it
>>>>> will pin (via vfio) all guest RAM. Could that be avoided using
>>>>> virtio-NVME, or what exactly would be the difference between virtio-nvme
>>>>> and ordinary NVME?
>>>>
>>>>
>>>> In the virtualization scenario where devices are offload to hardwareï
>>>>
>>>>
>>>> NVMEï
>>>> ---------------------------------------------------------------------------------------------------------------------
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂ _____________________________________________________________________________
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ___________________________________________________________ÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ _____________________________________________________Â |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ __________________________________ÂÂÂÂ | |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______
>>>> Â | | | User | | Mem |-----------------------|----|-|----|-----> | |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |______|ÂÂÂÂÂÂÂ SPDKÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ | (gVA) |______|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ (gVA)ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |______|___________________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |--GuestOS----------|--------------------------------| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______\/__________________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | VM |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ VFIOÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂ UserÂÂÂÂ |ÂÂÂ |ÂÂ KernelÂÂ |___________ÂÂÂÂ __________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>> Â | | | | vfio-pci | | | | | | | |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â (gIOVA)Â |ÂÂ | vfio_iommu_type1 |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>> Software | | | |______|____|___|__________________| | | | | |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |___________________|________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ ___________________|________________________________Â |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______|____ÂÂÂÂ __________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ \/ÂÂ \/
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂ \/ÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ NVMEÂÂ |ÂÂ |ÂÂÂÂÂ vIOMMUÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂ QEMUÂÂÂÂ |Â Instance |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ --|----|-|----|-----> |______|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â (gIOVA)Â |ÂÂ |ÂÂ (gIOVA-->gPA)Â |ÂÂÂ | |ÂÂÂ | (gPA)Â |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_____|_____|ÂÂ |__________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |__________________|_________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|___________________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |---HostOS---------------------------|----------------------------------------|ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ _______________________|___________________________________ÂÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ VFIOÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂ KernelÂÂ |_______________________|_____ÂÂÂÂ _________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ \/ÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂ vfio-pciÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂ vfio_iommu_type1ÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|_____|___|_________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |____________________________________|________________________________________|ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>> -----------------------------------------------|-------------------------------------------------|----|--------------
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂ ____________________________________\/____ÂÂÂÂ _________________________ÂÂÂÂÂÂÂÂÂÂ ___\/___\/___________
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂ |Â |ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂ |
>>>> Hardware | | | DMA (gIOVA) --|---|-> IOMMU --|---------|->|______| |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |_____________________________|ÂÂ |ÂÂÂÂÂ (gIOVA-->hPA)ÂÂÂÂÂ |ÂÂ (hPA) |ÂÂ Physical MemoryÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ | DPUÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |_________________________|ÂÂÂÂÂÂÂÂ |_____________________|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂÂÂ NVME-ofÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |__________________________________|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |___________________________|______________|
>>>> --------------------------------------|------------------------------------------------------------------------------
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | TCP (RDMA, and so on)
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ ______________v__________
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>> Remote storageÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ Network StorageÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |_________________________|
>>>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>>> ---------------------------------------------------------------------------------------------------------------------
>>>>
>>>>
>>>> It is difficult to implement PCIe passthrough live migration.
>>
>>
>>ÂÂ
>>> Linux commit 115dcec65f61d53e25e1bed5e380468b30f98b14 ("vfio: Define
>>> device migration protocol v2") defines the VFIO migration API and it's
>>> implemented by several drivers in the kernel.
>>
>> Yes, this commit supports VFIO live migration, but the feature is a work in progress,
>> recent submission: https://lore.kernel.org/all/20230116141135.12021-10-avihaih@nvidia.com/
>>ÂÂ
>>
>>> Can you explain the difficulty of implementing PCIe passthrough live
>>> migration in more detail?
>> VFIO live migration requires IOMMU to support dirty page tracking. Currently,
>> no IOMMU device supports this feature. So, VFIO live migration will take a long time.
>> Detailed information referenceïhttps://www.qemu.org/docs/master/devel/vfio-migration.html
>>
>>
>>
>>
>>>> virtio-nvme:
>>>> ---------------------------------------------------------------------------------------------------------------------
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂ _____________________________________________________________________________
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ___________________________________________________________ÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ _____________________________________________________Â |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ __________________________________ÂÂÂÂ | |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______
>>>> Â | | | User | | Mem |-----------------------|----|-|----|-----> | |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |______|ÂÂÂÂÂÂÂ SPDKÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ | (gVA) |______|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ (gVA)ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |______|___________________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |--GuestOS----------|--------------------------------| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______v___________________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | VM |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ VFIOÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂ UserÂÂÂÂ |ÂÂÂ |ÂÂ KernelÂÂ |___________ÂÂÂÂ __________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>> Â | | | | vfio-pci | | | | | | | |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â (gIOVA)Â |ÂÂ | vfio_iommu_type1 |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>> Software | | | |______|____|___|__________________| | | | | |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |___________________|________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ ___________________|________________________________Â |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______v____ÂÂÂÂ __________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ vÂÂÂ v
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |virtio-NVME|ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â Instance |ÂÂ |ÂÂÂÂÂ vIOMMUÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂ QEMUÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ --|----|-|----|-----> |______|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | vhost-vdpa|ÂÂ |ÂÂ (gIOVA-->gPA)Â |ÂÂÂ | |ÂÂÂ | (gPA)Â |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |__(gIOVA)__|ÂÂ |__________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |__________________|_________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|___________________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |---HostOS---------------------------|----------------------------------------|ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ _______________________|___________________________________ÂÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ vDPAÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂ KernelÂÂ |_______________________v_____ÂÂÂÂ _________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ vdpa-deviceÂÂÂÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | (Virtual device)ÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|_____|___|_________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |____________________________________|________________________________________|ÂÂÂÂÂÂÂ |ÂÂÂ |
>>>> -----------------------------------------------|-------------------------------------------------|----|--------------
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂ ____________________________________v_____ÂÂÂÂ _________________________ÂÂÂÂÂÂÂÂÂÂ ___v____v____________
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂ |Â |ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂ |
>>>> Hardware | | | DMA (gIOVA) --|---|-> IOMMU --|---------|->|______| |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |_____________________________|ÂÂ |ÂÂÂÂÂ (gIOVA-->hPA)ÂÂÂÂÂ |ÂÂ (hPA) |ÂÂ Physical MemoryÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |Â DPUÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |_________________________|ÂÂÂÂÂÂÂÂ |_____________________|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂÂÂ virtio-nvme-ofÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |__________________________________|
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂ |___________________________|______________|
>>>> --------------------------------------|------------------------------------------------------------------------------
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | TCP (RDMA, and so on)
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ ______________v__________
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>> Remote storageÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ Network StorageÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>>>Â ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |_________________________|
>>>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>>> ---------------------------------------------------------------------------------------------------------------------
>>>> Based on the vDPA framework, it supports live migration.
>>
>>
>>ÂÂ
>>
>>> The two diagrams are quite similar. Did you want to highlight a
>>> difference between the two approaches in the diagram?
>> The biggest difference is the VFIO and vDPA frameworks. The vDPA (virtio data path acceleration) kernel framework
>> is a pillar in productizing the end-to-end vDPA solution and it enables NIC vendors to integrate their vDPA NIC kernel
>> drivers into the framework as part of their productization efforts.
>> Detailed information referenceïhttps://www.redhat.com/en/blog/introduction-vdpa-kernel-framework
>Â
>Â
>Note that vDPA is not solely software concept but also a hardware one.
>And with vDPA kernel support the device can talks to kernel I/O
>subsystem via virtio-vDPA bus drivers.
Yes, I'm not being precise enough. vDPA is created for virtio devices to be offloading to hardware. Hardware is also part of vDPA.
Â
Â
Â
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]