screenshotting, screencasting & unity8+mir

[Gerry Boland]

On 26/11/13 12:31, Daniel d'Andrada wrote:
> Hi,
> 
> On November 26th I (Unity8), Kevin Gunn, Chris Gagnon (Autopilot) and
> Alexandros Frantzis (Mir) had a meeting on the requirements and
> implementation of screenshotting and screencasting.
> 
> Chris told us that what autopilot really wants is screencasting (not
> screen shots) as it records all that happens during a test case and
> publishes the resulting video in case of failure.
> 
> Kevin came with the point that application developers already want
> screenshotting and screencasting and therefore the solution should also
> cater to them (as opposed to having a solution that works for autopilot
> now and for other/third-party applications to be done only later).
> 
> Thus considering those requirements and the future mir architecture
> (using qt scenegraph), that's the implementation we agreed upon:
> 
> Unity8 would provide a D-Bus service for clients (applications,
> autopilot) to request for a screencast or screenshot. In case of a
> screenshot, the requestor would just provide a filename (or directory)
> where the screenshot(s) should be placed and unity8 would do it. For
> screencasts unity8 would provide a shared memory area to the requestor
> that would be kept up to date with the contents of the screen. Access to
> that shared memory area would be controlled with a semaphore and a
> mutex. So the problem of handling codecs, recording and audio stream to
> go along with it, etc is left to the user (i.e. an actual screencasting
> application or tool).
> 
> D-Bus was chosen because access control to the screenshot/screencast
> feature can be easily implemented using D-Bus security policies.
> According to Alexandros, Mir probably won't have to be modified for
> screen[shotting|casting]. As a QQuickWindow will be controlling the
> composition now (qt-scenegraph approach), what would have to be done is
> calling glReadPixels right after the scene graph is rendered, which is
> essentially how "QImage QQuickWindow::grabWindow()" is implemented (if I
> understood its code correctly).
> 
> So that's where I'm heading with the implementation (didn't start yet).
> If someone has a better idea, sees a problem with this approach or
> otherwise has something to add, please advise.
> 
> Feedback already received so far:
> 
> Thomi Richards:
>  - Clarified that autopilot also wants screenshots.
>  - Asked for a command line tool that does all the work of talking to
> that proposed screencasting API and outputs an ogv.
> 
> Michał Sawicz:
>  - "I'm not sold on the idea that it's unity8's / unity-mir's
> responsibility to provide that interface, though, as the ability to do
> those feels like a very generic need that should be built into the
> display server itself. Maybe the fact that we might be using Qt's
> scenegraph changes something here, though - but maybe that just means
> there's a new project growing on unity-mir's side?"
> 

Hey,
The system compositor will probably not be using the Qt scenegraph, but
instead Mir's own compositor. I don't know if using
QQuickWindow::grabWindow() will work in that case (though if it just
calls glReadPixels, it probably will).

Also if hardware (non-GL) compositing is being used, reading back pixels
via glReadPixels won't be enough as not everything on screen will be
drawn by GL.

As a result, I'm not certain we can rely on QQuickWindow::grabWindow() /
glReadPixels, but would need something internal to Mir.
Thanks
-Gerry