[Bug 412647]

Alessandro Decina alessandro.d at gmail.com
Wed Mar 14 10:13:44 UTC 2012

Created attachment 605705
nsBuiltinDecoder* based implementation

tl;dr: I'm attaching a new, simpler patch that adds gst decoding
support to firefox, based on the recent(ish) gecko
nsBuiltinDecoder* interfaces. Hopefully this version will be
easier to review, merge and maintain.

I'm a gst developer and I deal with embedded devices a lot. I
recently started working on this patch as I'm interested in
porting b2g to new devices. Also more generally, i'd like to see
firefox do media playback as efficiently as possible on embedded
devices, making use of gst's hw support.

Two weeks ago, after the GStreamer SDK effort was announced
(http://is.gd/tteunv), I decided to pick up the existing firefox
gst patch.  After getting familiar with it and the gecko media
code, and after an half failed rebase attempt (content/media/ has
changed significantly and the patch has bitrotten a bit), I
decided to write a new gst decoder based on the recent
nsBuiltinDecoder* interfaces in gecko.

My approach is deliberately simpler than the older patch. My goal
was to just get hw decoding support through gst, delegating the
rest of the work (i/o, synchronization, rendering, state
management etc) to the existing media code like the WebM and OGG
decoders do.

The code is self contained and 99% of it is in the
nsGStreamerReader class in
content/media/gstreamer/nsGStreamerReader.cpp. In terms of lines
of code and complexity, it's comparable to nsWebMReader and

Internally it uses the following gst pipeline: appsrc !
decodebin2 name=d d. ! ffmpegcolorspace !
video/x-raw-yuv,format=I420 ! appsink d. ! audioconvert !
audioresample ! appsink.

In english: it feeds data from gecko to gstreamer using appsrc,
decodes using the decodebin2 element and feeds a/v back to
firefox using appsink elements.

The patch adds an --enable-gstreamer opt-in switch to configure.
When enabled, it currently takes over WebM and OGG decoding and
adds H264 support (see my closing note efore starting flames on
this please). Eventually i think i'd like to make it
takeover WebM and OGG decoding depending on runtime prefs.

I tested the patch on OSX. I've got accelerated h264 decoding
(supported by the gst applemedia plugin) working but it requires
a colorspace conversion currently, which I plan to fix soonish in
the plugin. After that, my goal is to make firefox mobile do hw
accelerated decoding on android using the gst-omx plugin. 

There are a few known bugs at this point, the major one being
that I need to implement efficient buffer allocation. Also the
the bytes-read, frames-parsed/decoded stats are not implemented
yet. Being my first firefox patch, there are probably also plenty
of stylistic fixes needed. Anyway, overall I think it's good for
a first review.

Please note: i don't want to get into the argument of whether
firefox should support h264 decoding or not. The patch currently
adds h264 decoding only so that I could test and demo accelerated
decoding on OSX. I think decoding using gst has merits regardless of
whether h264 ends up being supported or not. I'm more than
willing to remove h264 support from the patch if that makes it
easier to get it merged.

You received this bug notification because you are a member of Mozilla
Bugs, which is subscribed to Mozilla.

  Firefox is not able to play mp4 <video> tags

To manage notifications about this bug go to:

More information about the Ubuntu-mozillateam-bugs mailing list