Re: [virtio-dev] Re: [virtio-comment] Problems with VIRTIO-4 and writeback only disks

[Stefan Hajnoczi <stefanha@redhat.com>]

On Tue, Sep 17, 2013 at 03:03:47PM +0930, Rusty Russell wrote:
> James Bottomley <jbottomley@parallels.com> writes:
> > On Mon, 2013-09-16 at 16:50 +0930, Rusty Russell wrote:
> >> I think we should do the reverse:
> >> 1) If you negotiate VIRTIO_BLK_F_WCE, you need to flush (ie. it's
> >>    writeback).  Otherwise it's write-through.
> >> 2) If (and only if) you negotiate VIRTIO_BLK_F_WCE, you may also negotiate
> >>    VIRTIO_BLK_F_CONFIG_WCE, in which case you can toggle it.
> >> 
> >> But I'm not convinced that #2 should exist at all.  Does someone have a
> >> real use case?
> >
> > There is no use case in Linux.  We provide knobs to toggle the cache
> > type in the SCSI disk class, but the kernel doesn't do it by default, it
> > just runs with the cache type because the toggle can be unreliable.

The virtio_blk driver provides a cache_type sysfs attr that userspace
can toggle.  I guess this was inspired by how the SCSI disk class does
it.

> OK.  How about this as a strawman:
> 
> 1) Kill VIRTIO_BLK_F_CONFIG_WCE as a bad idea.  You either negotiate
>    WCE or not during device setup; no toggling.
> 
> 2) If device offers VIRTIO_BLK_F_WCE, and driver negotiates it:
>    - Completed writes should be persistent if guest crashes.
>    - No ordering guarantees are given except where flush command sent.
>    - Flush SHOULD push all preceeding writes to permanent storage.
>      (in the RFC SHOULD sense, which is weaker than MUST).
> 
> 3) If device does not offer VIRTIO_BLK_F_WCE, or driver doesn't
>    negotiate it:
>    - Completed writes should be persistent if guest crashes.
>    - No flush commands are supported.
>    - No guarantee about writes hitting permanent storage.
> 
> This pretty neatly divides it into complex and simple cases.  If you
> want more fine-grained, you know where to find virtio-scsi...

#3 is worse than what we had with VIRTIO_BLK_F_WCE semantics.  In order
to keep things simple you weakened the guarantees to the point where you
have to look at your hypervisor implementation instead of the virtio
standard.  We're trying to define standard so guests and hypervisors can
work together - undefined behavior doesn't further that goal, it
actually prevents virtio implementations from working universally.

When VIRTIO_BLK_F_WCE is not offered by the device or negotiated by the
guest it makes sense to guarantee that every write hits permanent
storage.

Stefan