Re: [PATCH v5 0/2] virtio-fs: add virtio file system device

["Dr. David Alan Gilbert" <dgilbert@redhat.com>]

* Michael S. Tsirkin (mst@redhat.com) wrote:
> Didn't look at the spec yet, but I have a question:
> 
> On Fri, Jul 26, 2019 at 10:53:36AM +0100, Stefan Hajnoczi wrote:
> > v5:
> >  * Explain multiqueue semantics: no ordering, identical functionality on each queue, one FUSE session state shared between all queues
> >  * Explain how the FUSE session is started with a FUSE_INIT request
> >  * Consistently use "submit" vs "made available" and "complete" vs "used" terminology [Michael]
> >  * Explain endianness [Michael]
> >  * Clarify hiprio vs normal queue usage [Michael]
> >  * Move SHOULD, MUST, etc wording into normative sections [Michael]
> >  * Mention that FUSE_INIT negotiated state needs to be transferred during live migration [Michael]
> >  * State that the DAX window is mapped with writeback caching like RAM [Michael]
> >  * Mention DAX window mapping alignment constraints (they are communicated via the FUSE protocol) [Michael]
> >  * Explain that FUSE_SETUPMAPPING fails when device resources are exhausted and that splitting mappings consumes resources too [Michael]
> >  * Clarify access rules to DAX window - only touch memory that has a mapping establised
> >  * Document that DAX data persistence is achieved via FUSE_FSYNC
> 
> 
> The following was raised during v4 review:
> 
> 	> > > > There is a practical problem that the QEMU process may hit the mmap
> 	> > > > limit and be unable to perform its own mmaps due to the DAX Window.  A
> 	> > > > limit must be enforced on the host so that QEMU's internal mmaps
> 	> > > > succeed.
> 	> > >
> 	> > > But number of mmaps are already limited by dax window size which is
> 	> > > controlled by virtiofsd. So all user has to do is start with smaller
> 	> > > dax window size if this ever becomes a concern.
> 	> >
> 	> > Yes, the DAX Window size sets a hard limit on the number of mappings
> 	> > (window_size / page_size).  I think we should still communicate a
> 	> > maximum number of mappings to provide more control for cases where the
> 	> > page size and DAX Window size don't produce a desirable number.
> 	> >
> 	> > Consider that window_size = 8 GB and page_size = 4 KB already yields
> 	> > 2,097,152 maximum mappings.  Oops, that number is too large!
> 	>
> 	> Ok, so that's something which will be in virtiofsd where maximum number
> 	> of outstanding can be configured by user and if we cross that limit,
> 	> FUSE_SETUPMAPPING will be returned with some error?
> 
> 	Yes, virtiofsd will enforce the limit.
> 
> 	The guest driver will also be aware of the limit via VIRTIO
> 	configuration space.
> 
> was this addressed?

In this version Stefan has marked SETUPMAPPING to say it can fail if out
of resources.

The tricky problem with specifying a limit is that the limit is more
likely to be global than to be per-device, and so specifying it in
either the virtio config space or (as looked more appropriate) fuse init
doesn't feel right.

IMHO we should never hit the limit in practical use, so it's really just
a matter of saying that it can fail and setting a large enough limit
that non-abusive drivers dont cause a problem.

Dave


> 
> > v4:
> >  * Clarify that there are no request ordering guarantees between requests in a
> >    single queue [vgoyal]
> >  * Add explanation of FUSE session endianness detection [dgilbert]
> > 
> > v3:
> >  * Remove notifications virtqueue, it's unimplemented and can be added when
> >    needed [Miklos]
> >  * Add Security Considerations and Live Migration Considerations sections
> >    [Michael]
> > v2:
> >  * Clean up core virtio file system device spec
> >  * Add DAX window
> > 
> > These patches add the virtio file system device, which is based on Linux FUSE
> > but includes the DAX window extension.  Similar to virtio-scsi, which
> > transports SCSI commands, virtio-fs transports FUSE requests and the protocol
> > documentation is not duplicated here.
> > 
> > The DAX window allows file contents to be accessed directly from shared memory.
> > This eliminates copying of data, reduces the number of vmexits, and reduces the
> > guest's memory footprint.  It also allows coherent mmap MAP_SHARED semantics
> > between guests on the same host.
> > 
> > Stefan Hajnoczi (2):
> >   content: add virtio file system device
> >   virtio-fs: add DAX window
> > 
> >  content.tex      |   1 +
> >  introduction.tex |   3 +
> >  virtio-fs.tex    | 259 +++++++++++++++++++++++++++++++++++++++++++++++
> >  3 files changed, 263 insertions(+)
> >  create mode 100644 virtio-fs.tex
> > 
> > -- 
> > 2.21.0
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK