[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: Re: [virtio-comment] About adding a new device type virtio-nvme
On Tue, 17 Jan 2023 10:34:09 -0500, Stefan wrote:
>On Tue, Jan 17, 2023 at 05:41:57PM +0800, äèä wrote:
>> On Tue, 17 Jan 2023 09:32:05 +0100ïDavid wroteï
>> >On 17.01.23 03:04, äèä wrote:
>>
>>
>>
>> >> virtio-nvmeÂadvantages :
>> >> 1)Â live migration
>> >> 2)Â support remote storage
>>
>>
>>
>> >At least 1) is an implementation detail in the NVME implementation in
>> >the hypervisor. I suspect 2) in a similar way, or is there a fundamental
>> >issue with that?
>>
>>
>>
>> >One problematic thing about the NVME implementation in QEMU is that it
>> >will pin (via vfio) all guest RAM. Could that be avoided using
>> >virtio-NVME, or what exactly would be the difference between virtio-nvme
>> >and ordinary NVME?
>>
>>
>>
>> In the virtualization scenario where devices are offload to hardwareï
>>
>>
>> NVMEï
>> ---------------------------------------------------------------------------------------------------------------------
>>ÂÂÂÂÂÂÂÂÂÂÂ _____________________________________________________________________________ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ___________________________________________________________ÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ _____________________________________________________Â |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ __________________________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> | | | User | | Mem |-----------------------|----|-|----|-----> | |ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |______|ÂÂÂÂÂÂÂ SPDKÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ | (gVA) |______|ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ (gVA)ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |______|___________________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |--GuestOS----------|--------------------------------| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______\/__________________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | VM |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ VFIOÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂ UserÂÂÂÂ |ÂÂÂ |ÂÂ KernelÂÂ |___________ÂÂÂÂ __________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> | | | | vfio-pci | | | | | | | |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â (gIOVA)Â |ÂÂ | vfio_iommu_type1 |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> Software | | | |______|____|___|__________________| | | | | |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |___________________|________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ ___________________|________________________________Â |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______|____ÂÂÂÂ __________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ \/ÂÂ \/ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂ \/ÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ NVMEÂÂ |ÂÂ |ÂÂÂÂÂ vIOMMUÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂ QEMUÂÂÂÂ |Â Instance |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ --|----|-|----|-----> |______|ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â (gIOVA)Â |ÂÂ |ÂÂ (gIOVA-->gPA)Â |ÂÂÂ | |ÂÂÂ | (gPA)Â |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_____|_____|ÂÂ |__________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |__________________|_________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|___________________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |---HostOS---------------------------|----------------------------------------|ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ _______________________|___________________________________ÂÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ VFIOÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂ KernelÂÂ |_______________________|_____ÂÂÂÂ _________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ \/ÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂ vfio-pciÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂ vfio_iommu_type1ÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|_____|___|_________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |____________________________________|________________________________________|ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> -----------------------------------------------|-------------------------------------------------|----|--------------
>>ÂÂÂÂÂÂÂÂÂÂÂ ____________________________________\/____ÂÂÂÂ _________________________ÂÂÂÂÂÂÂÂÂÂ ___\/___\/___________Â
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂ |Â |ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂ |
>> Hardware | | | DMA (gIOVA) --|---|-> IOMMU --|---------|->|______| |
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |_____________________________|ÂÂ |ÂÂÂÂÂ (gIOVA-->hPA)ÂÂÂÂÂ |ÂÂ (hPA) |ÂÂ Physical MemoryÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂ | DPUÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |_________________________|ÂÂÂÂÂÂÂÂ |_____________________|
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂÂÂ NVME-ofÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |__________________________________|
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂ |___________________________|______________|
>> --------------------------------------|------------------------------------------------------------------------------
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | TCP (RDMA, and so on)
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ ______________v__________
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>> Remote storageÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ Network StorageÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |_________________________|
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> ---------------------------------------------------------------------------------------------------------------------
>>
>>
>> It is difficult to implement PCIe passthrough live migration.
Â
>Linux commit 115dcec65f61d53e25e1bed5e380468b30f98b14 ("vfio: Define
>device migration protocol v2") defines the VFIO migration API and it's
>implemented by several drivers in the kernel.
Yes, this commit supports VFIO live migration, but the feature is a work in progress,
recent submission: https://lore.kernel.org/all/20230116141135.12021-10-avihaih@nvidia.com/
>Can you explain the difficulty of implementing PCIe passthrough live
>migration in more detail?
VFIO live migration requires IOMMU to support dirty page tracking. Currently,
no IOMMU device supports this feature. So, VFIO live migration will take a long time.
Detailed information referenceïhttps://www.qemu.org/docs/master/devel/vfio-migration.html
>> virtio-nvme:
>> ---------------------------------------------------------------------------------------------------------------------
>>ÂÂÂÂÂÂÂÂÂÂÂ _____________________________________________________________________________ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ___________________________________________________________ÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ _____________________________________________________Â |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ __________________________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> | | | User | | Mem |-----------------------|----|-|----|-----> | |ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |______|ÂÂÂÂÂÂÂ SPDKÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ | (gVA) |______|ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ (gVA)ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |______|___________________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |--GuestOS----------|--------------------------------| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______v___________________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | VM |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ VFIOÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂ UserÂÂÂÂ |ÂÂÂ |ÂÂ KernelÂÂ |___________ÂÂÂÂ __________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> | | | | vfio-pci | | | | | | | |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â (gIOVA)Â |ÂÂ | vfio_iommu_type1 |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> Software | | | |______|____|___|__________________| | | | | |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |___________________|________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ ___________________|________________________________Â |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ ______v____ÂÂÂÂ __________________ÂÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ vÂÂÂ vÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |virtio-NVME|ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ ______ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |Â Instance |ÂÂ |ÂÂÂÂÂ vIOMMUÂÂÂÂÂ |ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂ QEMUÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ --|----|-|----|-----> |______|ÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | vhost-vdpa|ÂÂ |ÂÂ (gIOVA-->gPA)Â |ÂÂÂ | |ÂÂÂ | (gPA)Â |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |__(gIOVA)__|ÂÂ |__________________|ÂÂÂ | |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |__________________|_________________________________| |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|___________________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |---HostOS---------------------------|----------------------------------------|ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂ _______________________|___________________________________ÂÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ vDPAÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂ KernelÂÂ |_______________________v_____ÂÂÂÂ _________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ vdpa-deviceÂÂÂÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ | (Virtual device)ÂÂÂÂÂ |ÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂ |_______________________|_____|___|_________________________|ÂÂÂ |ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>>ÂÂÂÂÂÂÂÂÂÂ |____________________________________|________________________________________|ÂÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> -----------------------------------------------|-------------------------------------------------|----|--------------
>>ÂÂÂÂÂÂÂÂÂÂÂ ____________________________________v_____ÂÂÂÂ _________________________ÂÂÂÂÂÂÂÂÂÂ ___v____v____________Â
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂ |Â |ÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂ |
>> Hardware | | | DMA (gIOVA) --|---|-> IOMMU --|---------|->|______| |
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂ |_____________________________|ÂÂ |ÂÂÂÂÂ (gIOVA-->hPA)ÂÂÂÂÂ |ÂÂ (hPA) |ÂÂ Physical MemoryÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂ |Â DPUÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂ |_________________________|ÂÂÂÂÂÂÂÂ |_____________________|
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |ÂÂÂÂÂÂÂ virtio-nvme-ofÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂ |__________________________________|
>>ÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂ |___________________________|______________|
>> --------------------------------------|------------------------------------------------------------------------------
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ | TCP (RDMA, and so on)
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ ______________v__________
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>> Remote storageÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂ Network StorageÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ |_________________________|
>>ÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂÂ
>> ---------------------------------------------------------------------------------------------------------------------
>> Based on the vDPA framework, it supports live migration.
Â
>The two diagrams are quite similar. Did you want to highlight a
>difference between the two approaches in the diagram?
The biggest difference is the VFIO and vDPA frameworks. The vDPA (virtio data path acceleration) kernel framework
is a pillar in productizing the end-to-end vDPA solution and it enables NIC vendors to integrate their vDPA NIC kernel
drivers into the framework as part of their productization efforts.Â
Detailed information referenceïhttps://www.redhat.com/en/blog/introduction-vdpa-kernel-framework
Â
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]