The case for a "SceneGraph"

[Daniel van Vugt]

Sounds quite reasonable and agrees with some thoughts I've had too.

If there is anything I totally disagree with, I'm too tired to see it 
right now.

I think we have a good starting point now that compositor::Scene has 
landed today. However I would only add one note; that is to be careful 
to not move any significant parts around until you need to. This is code 
that a few of us are actively working on, and we want to minimize conflicts.

- Daniel


On 18/07/13 18:27, Alan Griffiths wrote:
> Hi All,
>
> at a recent hangout I took the action to produce some notes on what a
> "SceneGraph" means for Mir.
>
> We've had frequent discussions along the lines of "this information
> belongs in the SceneGraph, but as we don't have one we'll stick it
> <somewhere>". This, so far mythical, SceneGraph fulfills the "model"
> role in our core MVC pattern (see design/Architecture.dia in the source
> tree) and incorporates functionality currently present in a number of
> parts of the system.
>
> The present SurfaceStack implementation is a primitive SceneGraph that
> was "the simplest thing that might work" when it was added, but we've
> worked around its limitations for long enough to know that it is
> simplistic and fails to support the way we want the system to grow. The
> SurfaceController was initially intended as the Controller, but I think
> it is now clear that the Controller should be a part of the shell.
>
> So, what do we want from a SceneGraph?
>
> Looking at the architecture diagram and the code (which are in
> remarkable accord) there are a number of clients to the SceneGraph:
>
> *The compositor:*
>
> This needs access to the buffers to be composited and corresponding meta
> information about the way they are to be rendered. It doesn't happen now
> (the compositor has logic to determine which buffers to render), but the
> SceneGraph should be supplying exactly those buffers and meta
> information that the compositor needs.
>
> Requirement: the SceneGraph need to be able to send the buffers (and
> compositing metadata) that are visible on a specified output to a
> receiver within the compositor.
>
> An implication of this is that the SceneGraph needs to be able to model
> the output devices and know which surfaces are associated with each output.
>
> The current implementation has some simple logic that informs the
> compositor that changes have happened. We could make this more useful
> with a cleverer SceneGraph - e.g. identifying what has changed for which
> output.
>
> Side note: We currently implement visibility as a flag on each surface,
> but it might equally be a association within the SceneGraph - what is
> clear is that it should be part of the way that the SceneGraph passes
> buffers for rendering, not part of the metadata.
>
> *Input:*
>
> Input needs to determine where to route events based upon the input
> aware areas presented by some surfaces. The code here is undergoing a
> migration from an android based input stack to something more integrated
> into Mir.
>
> There are two scenarios: keyboard events (which go to the current input
> focus) and events that specify a point for which a corresponding input
> target has to be found.
>
> Currently the shell conspire with input to keep track of the input
> target. (The shell, in its role as Controller, should control the focus,
> but the focus information should be in the model - i.e. the SceneGraph.)
>
> I'm not sure of the exact state of the mechanism for position based
> input events currently but, to avoid race conditions when the SceneGraph
> changes, it should be the SceneGraph that identifies candidate input
> targets based on position.
>
> *The Shell:*
>
> As the Controller the shell should be mediating all updates to the
> SceneGraph - it gets to decide the policy for surface placement,
> ordering, focus changes, etc. This is best done by taking the raw
> request and posting the corresponding updates to the SceneGraph.
>
>  From the point of view of the SceneGraph we don't need to distinguish
> between the Mir "shell" code and the Unity shell code (the requirements
> on the SceneGraph should be the same). In practical terms a lot of
> interactions should stay on the C++ side and these probably best fit in
> the Mir codebase.
>
> Some examples:
>
> Surface creation: the shell needs to know when sessions create surfaces,
> and have control over where they are placed. Currently the Mir
> ApplicationSession receives the create_surface from frontend and has
> control of the request passed to the SurfaceController: we have two
> objects where we need one.
>
> Session creation: this is currently modelled in shell.
>
> Surface association: the shell (and possibly other clients) needs to be
> able to navigate between a session and its associated surfaces. The
> current SceneGraph has no concept of "session" - this relationship is
> currently held within ApplicationSession: this gives ApplicationSession
> a mixture of Controller and Model responsibilities.
>
> The SceneGraph should have a concept of "session" as there are
> operations it should perform on all the surfaces associated with the
> session.
>
> DepthID, z-order, etc. The stuff I've seen implemented around these seem
> doesn't leave me with a clear perception of the problem being solved -
> maybe someone else can fill out some details. There clearly is a need
> for an ordering to the buffers presented for rendering on each output
> but, so far, I don't see the need for the complexity we have.
>
> *Summary:*
>
> We need an improved Model and Controller for our SceneGraph - and that
> the model should support at least three types of node: sessions,
> surfaces and outputs (and understand the relationship between them.)
>
> The representation of outputs in the SceneGraph implies that there is
> more interaction with the display configuration than at present. (N.b.
> any updates to the display configuration should be mediated by the
> Controller/shell.)
>
> We're currently distributing knowledge of our "scene" around compositor,
> shell, and input. That is going to cause synchronization issues - we
> should move all this knowledge into one place.
>
> The above is just a first, high-level, stab at outlining the
> requirements - we clearly need to agree the scope and then drill down
> into a lot more detail as the next piece of work.
>
> Alan
>
>