New Buffer Semantics Planning

Kevin DuBois kevin.dubois at canonical.com
Mon Jul 20 19:28:27 UTC 2015


Some further planning notes (also a repeat of what I chatted about in the
us/euro standup about https://trello.com/c/I8s7W2f9)

I proposed these 3 branches today:
https://code.launchpad.net/~kdub/mir/multimonitor-arbiter/+merge/265301
https://code.launchpad.net/~kdub/mir/dropping-schedule/+merge/265303
https://code.launchpad.net/~kdub/mir/queueing-schedule/+merge/265304

Hopefully its not too much of a stretch to see how these 3 classes can be
integrated. The code isn't used yet, apart from the tests. Not all the
features in the spreadsheet have been implemented yet; but it seemed better
for the purposes of ongoing feedback on the code to keep the reviews small.
Once all the serverside and client side features have been covered, then it
can be switched to default (which https://trello.com/c/w8qfQ59p is a
placeholder for)

Thanks,
Kevin

On Mon, Jun 29, 2015 at 10:34 AM, Kevin DuBois <kevin.dubois at canonical.com>
wrote:

> So, as far as planning goes....
>
> I think that the spreadsheet (with a few nuances, perhaps) plus the
> BufferQueue.* tests give us a good capture of the requirements.
>
> So, seems sensible to link the translation of the BufferQueue tests and
> the capture of the requirements into the new integration-level test
> somehow. I'll see if I can track the translations in a spreadsheet somehow,
> so people can see more easily that the new test suite maps to the old one
> and the listed requirements.
>
>
>
> On Fri, Jun 26, 2015 at 3:22 AM, Daniel van Vugt <
> daniel.van.vugt at canonical.com> wrote:
>
>> TL;DR email is messy
>>
>> See this instead:
>> https://docs.google.com/spreadsheets/d/1qf3IssN_sujygMPK2sTX2AUhGiDUBF3TMYYcSRSaSy0/pubhtml
>>
>>
>>
>> On 26/06/15 12:24, Daniel van Vugt wrote:
>>
>>> That's why it's a hard problem to replace BufferQueue -- you have to
>>> figure out whether 2, 3 or 4 buffers for any given surface is correct at
>>> the time. Never over-allocate and never under-allocate (which causes
>>> freezing/deadlocks).
>>>
>>> What I was suggesting is that a frame callback (for non-idle surfaces
>>> anyway) would allow us to solve most of the hard problems in a
>>> distributed manner between the client and server processes fairly
>>> elegantly. And we don't need heuristics; this is all based on certainty
>>> and logical reasoning.
>>>
>>>
>>> On 26/06/15 12:16, Christopher James Halse Rogers wrote:
>>>
>>>>
>>>>
>>>> On Fri, Jun 26, 2015 at 2:04 PM, Daniel van Vugt
>>>> <daniel.van.vugt at canonical.com> wrote:
>>>>
>>>>> Hmm, maybe not. If we assume the server is only communicating with the
>>>>> client at 60Hz then the client could just do all the dropping itself
>>>>> and
>>>>> send one frame (the newest completed one) every 16.6ms when the server
>>>>> asks.
>>>>>
>>>>
>>>> I don't think the server is ever going to ask for a frame.
>>>>
>>>> All the client sees when the server is done with a buffer is that one of
>>>> their previously submitted buffers changes state from read-only to
>>>> exclusive-write access. It could possibly use a “has my last submitted
>>>> buffer become writeable yet” heuristic to guess when the server will
>>>> actually use a new buffer, but we don't guarantee that (nor can we, as
>>>> you note with bypass).
>>>>
>>>>
>>>>> On 26/06/15 12:01, Daniel van Vugt wrote:
>>>>> > bypass/overlays: If you look at the current logic you will see that
>>>>> the
>>>>> > DisplayBuffer holds the previous bypass/overlay buffer until _after_
>>>>> the
>>>>> > client has provided the next one. And it must, to avoid scan-out
>>>>> > artefacts. So the server holds two of them very briefly. But only
>>>>> one is
>>>>> > held most of the time. Without "predictive bypass" as I'm working on
>>>>> > right now, that buffer is held for almost two frames. With
>>>>> "predictive
>>>>> > bypass" it's closer to (but greater than still) one frame held. On
>>>>> > startup, absolutely you're right that only one buffer is required to
>>>>> get
>>>>> > bypass/overlays going. So my wording was wrong.
>>>>>
>>>>
>>>> Right, but that's fine. If the client has submitted one buffer, and is a
>>>> candidate for overlay, then it's clear that the old scanout buffer
>>>> *wasn't* from the client. We hold onto the old scanout buffer and start
>>>> scanning out of the (single) buffer the client has submitted.
>>>>
>>>> When the client submits a second buffer, the first isn't released until
>>>> we know it's no longer being scanned out of, but we don't need to have
>>>> the client's second buffer before scanning out of the first.
>>>>
>>>> We don't need to have two buffers around all the time for overlay; we
>>>> need to have two buffers around to *switch* overlay buffers. But the
>>>> fact that we're switching buffers already means that we've got at least
>>>> two buffers.
>>>>
>>>> This is sort of client-visible behaviour because the client can *see*
>>>> that the server is holding more than one buffer, but it's the same logic
>>>> for the client - “Do I have write ownership of a buffer? Yes: render to
>>>> it. No: wait¹ for one of my buffers to become writeable, or allocate a
>>>> new one”.
>>>>
>>>> ¹: Potentially “wait” by adding the fence to the GL command stream and
>>>> submit rendering commands anyway.
>>>>
>>>>
>>>>> >
>>>>> > client wake-up: I may have worded that poorly too. The point is in
>>>>> the
>>>>> > new world (tm) frame dropping mostly happens in the client (as
>>>>> opposed
>>>>> > to all in the server like it is today). But some of it still needs to
>>>>> > happen in the server because you don't want a compositor that tries
>>>>> to
>>>>> > keep up with a 1000 FPS client by scheduling all of those frames on a
>>>>> > 60Hz display. It has to drop some.
>>>>> >
>>>>> >
>>>>> > On 26/06/15 11:39, Christopher James Halse Rogers wrote:
>>>>> >> On Fri, Jun 26, 2015 at 12:39 PM, Daniel van Vugt
>>>>> >> <daniel.van.vugt at canonical.com> wrote:
>>>>> >>> I'm curious (but not yet concerned) about how the new plan will
>>>>> deal
>>>>> >>> with the transitions we have between 2-3-4 buffers which is neatly
>>>>> >>> self-contained in the single BufferQueue class right now.
>>>>> Although as
>>>>> >>> some responsibilities clearly live on one side and not the other,
>>>>> >>> maybe things could become conceptually simpler if we manage them
>>>>> >>> carefully:
>>>>> >>>
>>>>> >>>   framedropping: Always implemented in the client process as a
>>>>> >>> non-blocking acquire. The server just receives new buffers quicker
>>>>> >>> than usual and needs the smarts to deal with (skip) a high rate of
>>>>> >>> incoming buffers [1].
>>>>> >>
>>>>> >> Clients will need to tell the server at submit_buffer time whether
>>>>> or
>>>>> >> not this buffer should replace the other buffers in the queue.
>>>>> Different
>>>>> >> clients will need different behaviour here - the obvious case being
>>>>> a
>>>>> >> video player that wants to dump a whole bunch of time-stamped
>>>>> buffers on
>>>>> >> the compositor at once and then go to sleep for a while.
>>>>> >>
>>>>> >> But in general, yes. The client acquires a bunch of buffers and
>>>>> cycles
>>>>> >> through them.
>>>>> >>
>>>>> >>>   bypass/overlays: Always implemented in the server process,
>>>>> invisible
>>>>> >>> to the client. The server just can't enable those code paths
>>>>> until at
>>>>> >>> least two buffers have been received for a surface.
>>>>> >>
>>>>> >> I don't think that's the case? Why does the server need two
>>>>> buffers in
>>>>> >> order to overlay? Even with a single buffer the server always has a
>>>>> >> buffer available¹.
>>>>> >>
>>>>> >> It won't be entirely invisible to the client; we'll probably need
>>>>> to ask
>>>>> >> the client to reallocate buffers when overlay state changes, at
>>>>> least
>>>>> >> sometimes.
>>>>> >>
>>>>> >>>   client wake-up: Regardless of the model/mode in place the client
>>>>> >>> would get woken up at the physical display rate by the server if
>>>>> it's
>>>>> >>> had a buffer consumed (but not woken otherwise). More frequent
>>>>> >>> wake-ups for framedropping are the responsibility of libmirclient
>>>>> >>> itself and need not involve the server to do anything different.
>>>>> >>
>>>>> >> By and large, clients will be woken up by EGL when the relevant
>>>>> fence is
>>>>> >> triggered.
>>>>> >>
>>>>> >> I don't think libmirclient will have any role in waking the client.
>>>>> >> Unless maybe we want to mess around with
>>>>> >>
>>>>> >>> [1] Idea: If the server skipped/dropped _all_ but the newest
>>>>> buffer it
>>>>> >>> has for each surface on every composite() then that would eliminate
>>>>> >>> buffer lag and solve the problem of how to replace dynamic double
>>>>> >>> buffering. Client processes would still only be woken up at the
>>>>> >>> display rate so vsync-locked animations would not speed up
>>>>> >>> unnecessarily. Everyone wins -- minimal lag and maximal smoothness.
>>>>> >>
>>>>> >> ¹: The assumption here is that a buffer can be simultaneously
>>>>> scanned
>>>>> >> out from and textured from. I *think* that's a reasonable
>>>>> assumption,
>>>>> >> and in the cases where I know it doesn't apply having multiple
>>>>> buffers
>>>>> >> doesn't help, because it's the buffer *format* that can only be
>>>>> scanned
>>>>> >> out from, not textured from.
>>>>> >>
>>>>> >
>>>>>
>>>>> --
>>>>> Mir-devel mailing list
>>>>> Mir-devel at lists.ubuntu.com
>>>>> Modify settings or unsubscribe at:
>>>>> https://lists.ubuntu.com/mailman/listinfo/mir-devel
>>>>>
>>>>
>>>>
>> --
>> Mir-devel mailing list
>> Mir-devel at lists.ubuntu.com
>> Modify settings or unsubscribe at:
>> https://lists.ubuntu.com/mailman/listinfo/mir-devel
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.ubuntu.com/archives/mir-devel/attachments/20150720/a14b4982/attachment.html>


More information about the Mir-devel mailing list