What is a MirRenderSurface?

[Christopher James Halse Rogers]

On Thu, Sep 8, 2016 at 9:36 PM, Kevin DuBois 
<kevin.dubois at canonical.com> wrote:
> 
> 
> On Wed, Sep 7, 2016 at 8:01 PM, Christopher James Halse Rogers 
> <chris at cooperteam.net> wrote:
>> 
>> 
>> On Thu, Sep 8, 2016 at 5:09 AM, Cemil Azizoglu 
>> <cemil.azizoglu at canonical.com> wrote:
>>> Regardless ​of what EGL (or any other API) needs, there needs to 
>>> be a way for Mir platform to generically represent a unit of 
>>> renderable (hw or sw) with a distinct type. That's what 
>>> MirRenderSurface provides. "Generically" in the sense that it 
>>> doesn't matter what it's backed by (BS or PC or whatever), and it 
>>> can be passed around  (to other APIs, not just EGL) freely.
>> 
>> Right. There was a point (long ago) at which MirSurface was sort-of 
>> this atomic-unit-of-renderable, but with cursor-renderables and 
>> multiple-renderables-per-surface this went away.
>> 
>> It is useful to have an absolutely minimal ‘this is a thing that 
>> can have image content posted to it’ concept; MirRenderSurface 
>> should be that. (Indeed, I'm not sure that that MirRenderSurface 
>> should even have a *size*, and it definitely shouldn't have a pixel 
>> format; I've remarked as such on the prototype branches).
>> 
> 
> I'm on board here, right up until we try to cast MirRenderSurface to 
> EGLNativeWindowType.

This is what I mean by the Android platform being a poor model. We're 
*not* casting MirRenderSurface to EGLNativeWindowType. We're *defining* 
EGLNativeWindowType to be MirRenderSurface. We are the windowing 
system. We define what EGLNativeWindowType is.

Now, this is awkward on Android because we're pretending to be a 
different windowing system, SurfaceFlinger, but it's a quirk of the 
Android platform.

> (This is what I called "Option 2 in the OP). The 'absolutely minimal' 
> concept is a mf::BufferStreamId, and then the user has to tell us 
> what semantics they want to use to upload their content.
> 
> Being able to cast MirRenderSurface to differing native rendering 
> technologies:
> 1) Is not scalable. In the current spike we're really only casting to 
> EGLNativeWindowType, but will we cast to gstreamer handles? Can I 
> just make all the mir_presentation_chain() functions just take a 
> MirRenderSurface instead of a MirPresentationChain? (same with the 
> mir_buffer_stream() functions). Its really handy for EGL, but 
> unnecessarily ties all other ways of using Mir to EGL.

Again, this *doesn't* tie Mir to EGL. We're not casting 
MirRenderSurface to EGLNativeWindowType anymore than you cast Window* 
to EGLNativeWindowType on X11. Likewise, we're not tying 
MirRenderSurface to EGL anymore than Window* is tied to EGL on X11.

It *does* have implications for how we implement MirRenderSurface on 
Android; specifically, it means that MirRenderSurface needs to be a 
SurfaceFlinger window. But even this doesn't tie Mir to EGL here - if 
MirRenderSurface can masquerade as a SurfaceFlinger window it should 
also work for Android Vulcan, which expects a SurfaceFlinger window, 
and *does* also work for the HW video encoding, which expects a 
SurfaceFlinger window.

> 2) Its not obvious how to use this. The casting to an 
> EGLNativeWindowType has to be pointed out in comment-headers, or 
> gleaned from reading example code.
> 
> 3) It puts MirPresentationChain and MirBufferStream as second-class 
> citizens. Having to create an EGLNativeWindowType first, and then 
> create a MirPresentationChain doesn't make sense (unless from the 
> vantage of favoring EGL to the detriment of the other technologies). 
> It is more work for us to create an EGLNativeWindowSurface up-front 
> (or via an even more complicated system of deferred type realization) 
> regardless of whether the user wants to use EGL or not.
> 
> So, by tweaking the header like this: (repost: 
> http://pastebin.ubuntu.com/23149827/)
> These problems are resolved, and internally and externally, it 
> becomes clearer what's going on.

This:

EGLNativeWindowType mir_render_surface_create_eglnativewindowtype(
    MirRenderSurface* render_surface,
    int physical_width, int physical_height,
    void *egldisplay, void* eglconfig); //see 
mir_connection_get_pixel_format()

is a really confusing function. We're passing in an EGLDisplay and an 
EGLConfig to get a Mir-defined-type, the only purpose of which is to 
pass to EGLCreateWindowSurface to create an EGLSurface? Why aren't we 
just returning an EGLSurface?

Put like that, I think this makes it clear what's wrong - EGL *is* the 
layer that takes a window-system specific type (here, MirRenderSurface) 
and presents it as something cross-platform. It makes no sense to take 
a window-system specific type, convert it into *another* EGL-specific 
window-system specific type, and then create a cross-platform EGL 
object from it.

> 
> 
>>> Whereas, if we pass around, say, a BS handle then we become 
>>> dependent on a backend that we may want to change or even deprecate 
>>> in the future.
>>> 
>>> Also, EGL doesn't impose platforms to have an EGL specific type. A 
>>> platform has its own type, and EGL is *made* to work with it. 
>>> Here's a comment from the EGL header :
>>> /* The types NativeDisplayType, NativeWindowType, and 
>>> NativePixmapType
>>>  * are aliases of window-system-dependent types, such as X Display 
>>> * or
>>>  * Windows Device Context. They must be defined in platform-specific
>>>  * code below. The EGL-prefixed versions of Native*Type are the same
>>>  * types, renamed in EGL 1.3 so all types in the API start with 
>>> "EGL".
>>> That means we should have one consistent type that may take 
>>> different forms depending on the backend. So saying "A 
>>> MirRenderSurface is an EGLNativeWindowType" is not the right way to 
>>> look at it. Because there is nothing EGL-specific about a 
>>> MirRenderSurface.
>>> 
>>> As you know, ANativeWindow in Android, for instance, is not 
>>> specific to EGL. It just represents a native window for Android for 
>>> s/w and h/w. MirRenderSurface should be considered the same. You 
>>> have a point in saying that we already expose clients with too much 
>>> info on BS & PC. Perhaps we shouldn't, but that's a separate 
>>> discussion.
>> 
>> Right. EGLNativeWindowType is defined by the platform as “a handle 
>> to the thing you can render to”.
>> 
>> This is why Mir's Android support is misleading. On the Android 
>> platform Mir pretends to be SurfaceFlinger, and this is obviously 
>> awkward, even though Google has deliberately made it possible.
>> 
>> But this really is going to be a special case. We're not going to 
>> pretend to be Wayland to get closed-source Mali support; we're not 
>> going to pretend to be X11 to get closed-source NVIDIA support, 
>> although we could (at the cost of significant madness) in both cases.
>> 
> 
> This is a second topic from the one I originally brought up (having 
> to do with that we can cast MirRenderSurface to different rendering 
> technologies types). This topic is more what do we provide to drivers 
> and how we define our responsibilities to the different platform. It 
> needs to be addressed, but I think clearing up the MirRenderSurface 
> wip header is step one. Maybe I'll diverge the email chain to talk 
> about Mir's relationship to EGL/drivers.