Re: [virtio-dev] Re: [Qemu-devel] [PATCH v3 0/3] Use of unique identifier for pairing virtio and passthrough devices...

[Cornelia Huck <cohuck@redhat.com>]

On Fri, 6 Jul 2018 18:07:00 +0300
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Fri, Jul 06, 2018 at 03:54:06PM +0200, Cornelia Huck wrote:
> > On Thu, 5 Jul 2018 17:49:10 -0700
> > Siwei Liu <loseweigh@gmail.com> wrote:
> >   
> > > On Wed, Jul 4, 2018 at 5:15 AM, Cornelia Huck <cohuck@redhat.com> wrote:  
> > > > On Tue, 3 Jul 2018 16:31:03 -0700
> > > > Siwei Liu <loseweigh@gmail.com> wrote:
> > > >    
> > > >> On Tue, Jul 3, 2018 at 7:52 AM, Cornelia Huck <cohuck@redhat.com> wrote:    
> > > >> > From my point of view, there are several concerns:
> > > >> > - This approach assumes a homogeneous pairing (same transport), and
> > > >> >   even more, it assumes that this transport is pci.    
> > > >>
> > > >> Not really.
> > > >>
> > > >> There could be some other place to define a generic (transport
> > > >> independent) virtio feature, whereas the data (group ID) can be stored
> > > >> in transport specific way. That generic virtio feature and the way to
> > > >> specify target transport to group with is yet to be defined. I don't
> > > >> see this patch is in conflict with that direction.    
> > > >
> > > > Sorry, but I really do not see how this is not pci-specific.
> > > >
> > > > - One of your components is a bridge. A transport does not necessarily
> > > >   have that concept, at least not in a way meaningful for this approach
> > > >   to work.    
> > > 
> > > Assuming we have a transport agnostic high-level virtio feature to
> > > indicate that the target type for the to-be-paired device is PCI, then
> > > we have to use the data stored in a PC bridge to pair the device. It
> > > does not associate transport of virtio itself with the type of target
> > > device, right. The introduction of PCI bridge is just for storing the
> > > group ID data for the PCI passthrough device case, while one may
> > > define and introduce other means to retrieve the info if target device
> > > type is not PCI e.g. a special instruction to get group ID on s390x
> > > zPCI.  
> > 
> > Well, zPCI *is* PCI, just a very quirky one. I'm not sure how upper
> > layers are supposed to distinguish that.
> > 
> > Is there really no existing PCI identifier that (a) the host admin
> > already knows and (b) the guest can retrieve easily?  
> 
> There is e.g. PCI Express serial number. However given that
> one use of the feature is for live migration, we really
> shouldn't use any ID from a real card.

With any "vfio vs. live migration" setup, we basically have two
different cases:
- Both source and target use the same physical device (device shared
  between logical partitions, device accessible from two different
  machines).
- We are dealing with two different devices that can be configured to
  look/act the same.

I have been mostly thinking about the first case, but the serial number
probably won't work with the second case.

> 
> > >   
> > > > - Even if we can introduce transport-specific ways for other
> > > >   transports, the bridge concept still means that the pairing cannot be
> > > >   cross-transport.    
> > > 
> > > Sorry, I did not get this. A bridge is PCI specific concept indeed,
> > > but the virtio iteself can be of other transport. Why it matters? I
> > > guess a higher-level feature is needed to define the target transport
> > > for the to-be-paired device. The group ID info can reside on the
> > > transport specific area on virtio itself though. E.g. for virtio-pci,
> > > get it on the vendor specific capability in the PCI configuration
> > > space.
> > > For virtio-ccw, you may come up with CCW specific way to get the group
> > > ID data. Is this not what you had in mind?  
> > 
> > No, my idea was it to add it as a field in the virtio-net config space,
> > making it transport-agnostic. (And making this a typed value, depending
> > on the vfio device we want to pair the virtio device with.)
> > 
> > If we want to extend that to other device types, we can add the field
> > in their config space; but I'd rather prefer a more generic "host
> > relays config metainformation" approach.
> >   
> > >   
> > > >
> > > > I think we should be clear what we want from a generic
> > > > (transport-agnostic) virtio feature first. Probably some way to relay
> > > > an identifier of the to-be-paired device (transport-specific +
> > > > information what the transport is?)
> > > >    
> > > Are we talking about the transport of the virtio itself or the target
> > > device? Note the virtio feature must always be retrieved using
> > > transport specific way, although the semantics of the feauture can be
> > > transport independent/agnostic. Once a virtio driver instace gets to
> > > the point to read feature bits from the device, the transport of that
> > > virtio device is determined and cannot be changed later.  
> > 
> > No, I don't want to introduce a new, per-transport mechanism, but
> > rather put it into the config space.  
> > > 
> > >   
> > > >> > - It won't work for zPCI (although zPCI is really strange) -- this
> > > >> >   means it will be completely unusable on s390x.    
> > > >> I still need more information about this use case. First off, does
> > > >> zPCI support all the hot plug semantics or functionality the same way
> > > >> as PCI? Or there has to be some platform or firmeware support like
> > > >> ACPI hotplug? Does QEMU have all the pieces ready for s390 zPCI
> > > >> hotplug?    
> > > >
> > > > zPCI is a strange beast, so first a pointer to a writeup I did:
> > > > https://virtualpenguins.blogspot.de/2018/02/notes-on-pci-on-s390x.html
> > > >
> > > > It does support hotplug, within the s390 architectural context, but
> > > > that should be fine for our needs here.
> > > >
> > > > My concern comes from the 'no topology' issue. We build a fake topology
> > > > in QEMU (to use the generic pci infrastructure), but the guest does not
> > > > see any of this. It issues an instruction and gets a list of functions.
> > > > This means any bridge information is not accessible to the guest.    
> > > 
> > > That is the reason why I prefer leaving it to specific transport
> > > (zPCI) to define its own means to present and retrieve the group ID
> > > information. The high-level feature bit just provides the indirection
> > > to the target transport (for the to-be-paired device), while seperate
> > > transport can have a way of its own to present/retrieve the group ID
> > > data.
> > > 
> > > I don't know where to present that group ID data on s390 zPCI though
> > > to be honest. But I doubt we can come up with a very general
> > > transport-agnostic way to present the data for both (and pontentially
> > > ALL) bus architectures.  
> > 
> > I don't want to establish zPCI as a different transport than 'normal'
> > PCI; that would just make life difficult for everyone. The 'put it into
> > virtio config space' approach would mean a second feature bit, but
> > should just work.  
> 
> So how about that. Each transport will gain a new field
> which is offset within config space of a generic config.
> Each field within that will be guarded by a feature bit.
> 
> E.g. add CCW_CMD_READ_CONF_OFFSET.
> 
> Is this better than a separate new generic config space?

From a ccw view, it would amount to the same (i.e., a new command is
needed in any case). Currently, I'm actually not quite sure how much
more designing of this does make sense (I'll reply elsewhere in this
thread).