The case for a "SceneGraph"

Gerry Boland gerry.boland at canonical.com
Thu Jul 18 23:58:25 UTC 2013


Hi Alan,
mind if I add some questions & comments from the shell PoV?

On 18/07/13 12:27, Alan Griffiths wrote:
> 
> So, what do we want from a SceneGraph?

What shell will need is, for every surface, the following properties:
- x, y, z, width, height
- opacity, visible
- scale
- ability to set more complex transformations/shaders, so can desaturate
surface colours, do more advanced animations, blurs etc.

Shell will implement window management through QML. I would like to have
as much flexibility with surfaces as the Qt Scenegraph gives me with
Items [1] [2].

> 
> Looking at the architecture diagram and the code (which are in
> remarkable accord) there are a number of clients to the SceneGraph:
> 
<snip>
> 
> *Input:*
> 
> Input needs to determine where to route events based upon the input
> aware areas presented by some surfaces. The code here is undergoing a
> migration from an android based input stack to something more integrated
> into Mir.
> 
> There are two scenarios: keyboard events (which go to the current input
> focus) and events that specify a point for which a corresponding input
> target has to be found.
> 
> Currently the shell conspire with input to keep track of the input
> target. (The shell, in its role as Controller, should control the focus,
> but the focus information should be in the model - i.e. the SceneGraph.)
> 
> I'm not sure of the exact state of the mechanism for position based
> input events currently but, to avoid race conditions when the SceneGraph
> changes, it should be the SceneGraph that identifies candidate input
> targets based on position.

I don't follow your last point. Could you give an example use-case please?

> 
> *The Shell:*
> 
> As the Controller the shell should be mediating all updates to the
> SceneGraph - it gets to decide the policy for surface placement,
> ordering, focus changes, etc. This is best done by taking the raw
> request and posting the corresponding updates to the SceneGraph.
> 
> From the point of view of the SceneGraph we don't need to distinguish
> between the Mir "shell" code and the Unity shell code (the requirements
> on the SceneGraph should be the same). In practical terms a lot of
> interactions should stay on the C++ side and these probably best fit in
> the Mir codebase.

This confused me too. Can you explain to me where the boundary line
between Mir and Unity shell code lies? What exactly are these
"interactions"?


> Some examples:
> 
> Surface creation: the shell needs to know when sessions create surfaces,
> and have control over where they are placed. Currently the Mir
> ApplicationSession receives the create_surface from frontend and has
> control of the request passed to the SurfaceController: we have two
> objects where we need one.
> 
> Session creation: this is currently modelled in shell.
> 
> Surface association: the shell (and possibly other clients) needs to be
> able to navigate between a session and its associated surfaces. The
> current SceneGraph has no concept of "session" - this relationship is
> currently held within ApplicationSession: this gives ApplicationSession
> a mixture of Controller and Model responsibilities.
> 
> The SceneGraph should have a concept of "session" as there are
> operations it should perform on all the surfaces associated with the
> session.

So I don't follow why the scenegraph needs this. Shell needs to know the
connection between surfaces and sessions, but scenegraph should just be
in charge of displaying surfaces. What use-cases do you see where shell
will need this extra info in SG?


> DepthID, z-order, etc. The stuff I've seen implemented around these seem
> doesn't leave me with a clear perception of the problem being solved -
> maybe someone else can fill out some details. There clearly is a need
> for an ordering to the buffers presented for rendering on each output
> but, so far, I don't see the need for the complexity we have.

+1 on this. Simple z-order should be sufficient IMO. I'm aware that
DepthID can help with surface ordering (keeps shell on top, sidestage
can stay on top of main stage), but I don't think it is worth the
trouble. I'd favour the simple API of a sole z-order property.


> 
> *Summary:*
> 
> We need an improved Model and Controller for our SceneGraph - and that
> the model should support at least three types of node: sessions,
> surfaces and outputs (and understand the relationship between them.)
> 
> The representation of outputs in the SceneGraph implies that there is
> more interaction with the display configuration than at present. (N.b.
> any updates to the display configuration should be mediated by the
> Controller/shell.)
> 
> We're currently distributing knowledge of our "scene" around compositor,
> shell, and input. That is going to cause synchronization issues - we
> should move all this knowledge into one place.
> 
> The above is just a first, high-level, stab at outlining the
> requirements - we clearly need to agree the scope and then drill down
> into a lot more detail as the next piece of work.


Thanks
-Gerry


[1]
https://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-item.html#transform-prop
[2] https://qt-project.org/doc/qt-5.0/qtquick/qml-qtquick2-shadereffect.html



More information about the Mir-devel mailing list