[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]
Subject: Re: [virtio-comment] [PATCH 00/11] Introduce transitional mmr pci device
On Fri, Mar 31, 2023 at 01:58:23AM +0300, Parav Pandit wrote:
> Overview:
> ---------
> The Transitional MMR device is a variant of the transitional PCI device.
What does "MMR" mean?
> It has its own small Device ID range. It does not have I/O
> region BAR; instead it exposes legacy configuration and device
> specific registers at an offset in the memory region BAR.
>
> Such transitional MMR devices will be used at the scale of
> thousands of devices using PCI SR-IOV and/or future scalable
> virtualization technology to provide backward
> compatibility (for legacy devices) and also future
> compatibility with new features.
>
> Usecase:
> --------
> 1. A hypervisor/system needs to provide transitional
> virtio devices to the guest VM at scale of thousands,
> typically, one to eight devices per VM.
>
> 2. A hypervisor/system needs to provide such devices using a
> vendor agnostic driver in the hypervisor system.
Is the idea that the hypervisor configures the new Transitional MMR
devices and makes them appear like virtio-pci Transitional devices?
In other words, the guest doesn't know about Transitional MMR and does
not need any code changes.
> 3. A hypervisor system prefers to have single stack regardless of
> virtio device type (net/blk) and be future compatible with a
> single vfio stack using SR-IOV or other scalable device
> virtualization technology to map PCI devices to the guest VM.
> (as transitional or otherwise)
What does this paragraph mean?
>
> Motivation/Background:
> ----------------------
> The existing transitional PCI device is missing support for
> PCI SR-IOV based devices. Currently it does not work beyond
> PCI PF, or as software emulated device in reality. It currently
> has below cited system level limitations:
>
> [a] PCIe spec citation:
> VFs do not support I/O Space and thus VF BARs shall not
> indicate I/O Space.
>
> [b] cpu arch citiation:
> Intel 64 and IA-32 Architectures Software Developerâs Manual:
> The processorâs I/O address space is separate and distinct from
> the physical-memory address space. The I/O address space consists
> of 64K individually addressable 8-bit I/O ports, numbered 0 through FFFFH.
>
> [c] PCIe spec citation:
> If a bridge implements an I/O address range,...I/O address range
> will be aligned to a 4 KB boundary.
>
> [d] I/O region accesses at PCI system level is slow as they are non-posted
> operations in PCIe fabric.
>
> The usecase requirements and limitations above can be solved by
> extending the transitional device, mapping legacy and device
> specific configuration registers in a memory PCI BAR instead
> of using non composable I/O region.
>
> Please review.
Modern devices were added to Linux in 2014 and support SR-IOV. Why is it
important to support Transitional (which really means Legacy devices,
otherwise Modern devices would be sufficient)?
>
> Patch summary:
> --------------
> patch 1 to 5 prepares the spec
> patch 6 to 11 defines transitional mmr device
>
> patch-1 uses lower case alphabets to name device id
> patch-2 move transitional device id in legay section along with
> revision id
> patch-3 splits legacy feature bits description from device id
> patch-4 rename and moves virtio config registers next to 1.x
> registers section
> patch-5 Adds missing helper verb in terminology definitions
> patch-6 introduces transitional mmr device
> patch-7 introduces transitional mmr device pci device ids
> patch-8 introduces virtio extended pci capability
> patch-9 describes new pci capability to locate legacy mmr
> registers
> patch-10 extended usage of driver notification capability for
> the transitional mmr device
> patch-11 adds conformance section of the transitional mmr device
>
> This design and details further described below.
>
> Design:
> -------
> Below picture captures the main small difference between current
> transitional PCI SR-IOV VF and transitional MMR SR-IOV VF.
>
> +------------------+ +--------------------+ +--------------------+
> |virtio 1.x | |Transitional | |Transitional |
> |SRIOV VF | |SRIOV VF | |MMR SRIOV VF |
> | | | | | |
> ++---------------+ | ++---------------+ | ++---------------+ |
> ||dev_id = | | ||dev_id = | | ||dev_id = | |
> ||{0x1040-0x106C}| | ||{0x1000-0x103f}| | ||{0x10f9-0x10ff}| |
> |+---------------+ | |+---------------+ | |+---------------+ |
> | | | | | |
> |+------------+ | |+------------+ | |+-----------------+ |
> ||Memory BAR | | ||Memory BAR | | ||Memory BAR | |
> |+------------+ | |+------------+ | || | |
> | | | | || +--------------+| |
> | | |+-----------------+ | || |legacy virtio || |
> | | ||IOBAR impossible | | || |+ dev cfg || |
> | | |+-----------------+ | || |registers || |
> | | | | || +--------------+| |
> | | | | |+-----------------+ |
> +------------------+ +--------------------+ +--------------------+
>
> Here transitional MMR SR-IOV VF has legacy configuration and
> legacy device specific registers located at an offset in the memory
> region BAR.
>
> A memory region can be dedicated at BAR0 or it can be in an
> existing BAR, allowing flexibility when implementing support
> in a hardware device.
>
> Transitional MMR SR-IOV VFs use a distinct device ID range to that
> of existing virtio SR-IOV VFs to allow flexibility in driver
> binding.
>
> A more zoom-in version of transitional MMR SR-IOV device shows
> that the location of the legacy registers are discovered by the
> driver using a new capability.
>
> +------------------------------+
> |Transitional |
> |MMR SRIOV VF |
> | |
> ++---------------+ |
> ||dev_id = | |
> ||{0x10f9-0x10ff}| |
> |+---------------+ |
> | |
> ++--------------------+ |
> || PCIe ext cap = 0xB | |
> || cfg_type = 10 | |
> || offset = 0x1000 | |
> || bar = N {0..5}| |
> |+--|-----------------+ |
> | | |
> | | |
> | | +-------------------+ |
> | | | Memory BAR = A | |
> | | | | |
> | +------>+--------------+ | |
> | | |legacy virtio | | |
> | | |+ dev cfg | | |
> | | |registers | | |
> | | +--------------+ | |
> | +-----------------+ | |
> +------------------------------+
>
> Software usage:
> ---------------
> Transitional MMR device can be used by multiple ways.
>
> 1. The most common way to use and map to the guest VM is by
> using vfio driver framework in Linux kernel.
>
> +----------------------+
> |pci_dev_id = 0x100X |
> +---------------|pci_rev_id = 0x0 |-----+
> |vfio device |BAR0 = I/O region | |
> | |Other attributes | |
> | +----------------------+ |
> | |
> + +--------------+ +-----------------+ |
> | |I/O to memory | | Other vfio | |
> | |rd/wr mapper | | functionalities | |
> | +--------------+ +-----------------+ |
> | |
> +-------------------+------------------------+
> |
> +------------+-----------------+
> | Transitional |
> | MMR SRIOV VF |
> +------------------------------+
>
> 2. Virtio pci driver to bind to the listed device id and
> use it as native device in the host.
>
> 3. Use it in a light weight hypervisor to run bare-metal OS.
>
> Parav Pandit (11):
> transport-pci: Use lowecase alphabets
> transport-pci: Move transitional device id to legacy section
> transport-pci: Split notes of PCI Device Layout
> transport-pci: Rename and move legacy PCI Device layout section
> introduction: Add missing helping verb
> introduction: Introduce transitional MMR interface
> transport-pci: Introduce transitional MMR device id
> transport-pci: Introduce virtio extended capability
> transport-pci: Describe PCI MMR dev config registers
> transport-pci: Use driver notification PCI capability
> conformance: Add transitional MMR interface conformance
>
> conformance.tex | 11 +-
> introduction.tex | 34 +++-
> tmmr-conformance.tex | 27 +++
> transport-pci.tex | 405 ++++++++++++++++++++++++++++++-------------
> 4 files changed, 354 insertions(+), 123 deletions(-)
> create mode 100644 tmmr-conformance.tex
>
> --
> 2.26.2
>
>
> This publicly archived list offers a means to provide input to the
> OASIS Virtual I/O Device (VIRTIO) TC.
>
> In order to verify user consent to the Feedback License terms and
> to minimize spam in the list archive, subscription is required
> before posting.
>
> Subscribe: virtio-comment-subscribe@lists.oasis-open.org
> Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
> List help: virtio-comment-help@lists.oasis-open.org
> List archive: https://lists.oasis-open.org/archives/virtio-comment/
> Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
> List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
> Committee: https://www.oasis-open.org/committees/virtio/
> Join OASIS: https://www.oasis-open.org/join/
>
Attachment:
signature.asc
Description: PGP signature
[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index] | [List Home]