Re: [virtio] New virtio balloon...

[Rusty Russell <rusty@au1.ibm.com>]

Luiz Capitulino <lcapitulino@redhat.com> writes:
> On Thu, 30 Jan 2014 12:16:29 +0200
> "Michael S. Tsirkin" <mst@redhat.com> wrote:
>
>> Also copy virtio-dev since this in clearly implementation ...
>> 
>> On Thu, Jan 30, 2014 at 07:34:30PM +1030, Rusty Russell wrote:
>> > Hi,
>> > 
>> >         I tried to write a new balloon driver; it's completely untested
>> > (as I need to write the device).  The protocol is basically two vqs, one
>> > for the guest to send commands, one for the host to send commands.
>> > 
>> > Some interesting things come out:
>> > 1) We do need to explicitly tell the host where the page is we want.
>> >    This is required for compaction, for example.
>> > 
>> > 2) We need to be able to exceed the balloon target, especially for page
>> >    migration.  Thus there's no mechanism for the device to refuse to
>> >    give us the pages.
>> > 
>> > 3) The device can offer multiple page sizes, but the driver can only
>> >    accept one.  I'm not sure if this is useful, as guests are either
>> >    huge page backed or not, and returning sub-pages isn't useful.
>> > 
>> > Linux demo code follows.
>> > 
>> > Cheers,
>> > Rusty.
>> 
>> More comments:
>> 	- for projects like auto-ballooning that Luiz works on,
>> 	  it's not nice that to swap page 1 for page 2
>> 	  you have to inflate then deflate
>> 	  besides overhead this confuses the host:
>> 	  imagine you tell QEMU to increase target,
>> 	  meanwhile guest inflates temporarily,
>> 	  QEMU thinks okay done, now you suddenly deflate.
>
> Yes. Just to give more context: one of my auto-ballooning versions broke
> when virtballoon_migratepage() ran. The reason was that my host-side code
> in the balloon device did not expect guest initiated operations. And the
> current spec does imply that all operations are initiated by the host.
>
> So, first suggestion: if the current spec is still valid, we have to add
> a note there that balloon operations can be initiated by the guest.

Or we could have an explicit exchange operation.  That may make more
sense?

> My current code is different, but something it does that could also brake
> due to guest initiated inflate/deflate is that it keeps track of the
> current balloon size. This is done by a counter which is incremented
> on inflate and decremented on deflate. I did that because the device just
> doesn't have this information ('actual' is unreliable, besides it's
> only updated every 256 pages inflated/deflated).
>
> Second suggestion: I think we need a reliable way to know the current
> balloon size on the host. My counter does work, btw.

Yes, this patch means we explicitly tell the host what we're doing with
the balloon, in the assumption that it keeps track.

> As far as the guest is concerned, my current code just informs the host
> that the guest is facing pressure. This is done through a "message" virtqueue,
> but I think this could just use the guest command virtqueue.

Yes, this rewrite has only two queues, a guest command queue (get pages,
give pages, need mem and stats reply) and a host command queue (set
target, get stats).

>> A couple of other suggestions:
>> 
>> - how to accomodate memory pressure in guest?
>>   Let's add a field telling host how hard do we
>>   want our memory back
>
> I agree we have to accommodate pressure in the guest some way, but what
> you proposed is more or less related to auto-ballooning.
>
> My suggestion would be for the host to tell the guest what to do in
> case of pressure. Like, it could tell the guest to just keep trying like
> it does today or it could ask the guest to stop inflation on pressure
> (which would require an ack from the host, which complicates the
> protocol a bit).

How about we allow the guest to send gratuitous stats?  (ie. not just
when the host asks).  The host should be able to tell from that whether
to change the target (rather than a simple need_mem).

If the host wants the guest to stop inflating, it should alter the
target.

> Also, there are two ways to know the guest is under pressure: 1. when
> alloc_page() fails or 2. use in-kernel vmpressure notification like
> auto-balloon does.

Both should probably warn the host.  We might need extra stats?

Cheers,
Rusty.