screenshotting, screencasting & unity8+mir
Daniel van Vugt
daniel.van.vugt at canonical.com
Wed Nov 27 08:00:50 UTC 2013
In fact, there's probably no reason why we can't just add a special
Such a surface would work in reverse for a single DisplayBuffer. You
could read from it by existing means (like
mir_surface_get_current_buffer) to get the framebuffer, but swapping it
just provides you with "the next composited frame". A readback surface
never affects screen contents, only returns them.
I think I prototyped something that worked like that when experimenting
in the early days of GBM bypass support.
The only significant challenge then is enforcing security. Deciding how
to authenticate a privileged client before it can
Regardless of the actual implementation details, I think we would be
silly to not utilize the existing buffer IPC code we already have.
On 27/11/13 09:41, Daniel van Vugt wrote:
> Another interesting thought...
> Mir already has the code to share buffers with external processes at
> high speed. Think about that for a while :)
> On 27/11/13 09:38, Daniel van Vugt wrote:
>> On an implementation note...
>> I think adding yet more library dependencies (dbus) to Mir would be a
>> mistake. It's already quite bloated in that respect.
>> The key recording functionality in Mir shouldn't care about the
>> communications anyway. So it would probably be a cleaner division of
>> responsibility to make the bus communication external to libmir*. Either
>> attached to unity-mir or some privileged client etc.
>> On 26/11/13 20:31, Daniel d'Andrada wrote:
>>> On November 26th I (Unity8), Kevin Gunn, Chris Gagnon (Autopilot) and
>>> Alexandros Frantzis (Mir) had a meeting on the requirements and
>>> implementation of screenshotting and screencasting.
>>> Chris told us that what autopilot really wants is screencasting (not
>>> screen shots) as it records all that happens during a test case and
>>> publishes the resulting video in case of failure.
>>> Kevin came with the point that application developers already want
>>> screenshotting and screencasting and therefore the solution should also
>>> cater to them (as opposed to having a solution that works for autopilot
>>> now and for other/third-party applications to be done only later).
>>> Thus considering those requirements and the future mir architecture
>>> (using qt scenegraph), that's the implementation we agreed upon:
>>> Unity8 would provide a D-Bus service for clients (applications,
>>> autopilot) to request for a screencast or screenshot. In case of a
>>> screenshot, the requestor would just provide a filename (or directory)
>>> where the screenshot(s) should be placed and unity8 would do it. For
>>> screencasts unity8 would provide a shared memory area to the requestor
>>> that would be kept up to date with the contents of the screen. Access to
>>> that shared memory area would be controlled with a semaphore and a
>>> mutex. So the problem of handling codecs, recording and audio stream to
>>> go along with it, etc is left to the user (i.e. an actual screencasting
>>> application or tool).
>>> D-Bus was chosen because access control to the screenshot/screencast
>>> feature can be easily implemented using D-Bus security policies.
>>> According to Alexandros, Mir probably won't have to be modified for
>>> screen[shotting|casting]. As a QQuickWindow will be controlling the
>>> composition now (qt-scenegraph approach), what would have to be done is
>>> calling glReadPixels right after the scene graph is rendered, which is
>>> essentially how "QImage QQuickWindow::grabWindow()" is implemented (if I
>>> understood its code correctly).
>>> So that's where I'm heading with the implementation (didn't start yet).
>>> If someone has a better idea, sees a problem with this approach or
>>> otherwise has something to add, please advise.
>>> Feedback already received so far:
>>> Thomi Richards:
>>> - Clarified that autopilot also wants screenshots.
>>> - Asked for a command line tool that does all the work of talking to
>>> that proposed screencasting API and outputs an ogv.
>>> Michał Sawicz:
>>> - "I'm not sold on the idea that it's unity8's / unity-mir's
>>> responsibility to provide that interface, though, as the ability to do
>>> those feels like a very generic need that should be built into the
>>> display server itself. Maybe the fact that we might be using Qt's
>>> scenegraph changes something here, though - but maybe that just means
>>> there's a new project growing on unity-mir's side?"
More information about the Mir-devel