Welcome back to "How to build a movie player". This is second lesson (lesson?). This time we'll aproach GStreamer to begin understanding how it works and what need to be done to build the multimedia subsystem of our player. First of all, some URL. GStreamer site is located at http://gstreamer.freedesktop.org/. On that site you'll find a lot of documentation to better focus arguments exposed here.

GStreamer is a framework, composed by several different reusable components called plugins. Plugins can be of more than one kind: sources, demuxers, decoders, video and audio players. We mentioned demuxes and decoders only because we are building a player application, but GStreamer, as the name suggest, is also able to produce contents, so it's also equipped with muxers, encoders and network stremers.

Plugins are interconnected by pads. Pads are like directional slots that produce or consume data. Some plaugins have just producing pads, like file readers or stream downloaders, some have consuming pads only, like audio or video players, some have both, like filtering plugins, demuxes, decoders and so on. Producing pads are called sources, consuming pads are called syncs.

Joining plugins by connecting pads creates a pipeline, which is a complete chain of media management able to acquire, process and output audio and video contents. If you need total control of what's happening inside your application, you can build your own pipeline by yourself. Otherwise you can follow GStreamer developer suggestion and use the prebuild playbin pipeline. Playbin is a plugin providing a complete pipeline which needs very little customization to work. Playing media content requires just setting the uri property of playbin to point to your file or stream and playbin will handle all the dirt work for you.

GStreamer comes with two useful command line tools: gst-launch and gst-inspect. The first one can be used to test pipelines before writing even one single line of code. The second one can be used to inspect plugin properties and found the plugin that best matches your needs. For example, the man page of gst-launch report some examples of pipelines, like the following:

gst-launch filesrc location=music.mp3 ! mad ! audioconvert ! audioresample ! osssink

Reading for left to right, this pipeline does:

1. read file music.mp3 with filesrc plugin
2. process it with mad plugin to decode the MP3 stream
3. convert it to a playable audio format with audioconvert plugin
4. resample to a useful rate with audioresample plugin
5. play it to /dev/dsp using the osssink plugin, which is based on OSS Linux audio system

location is a property of filesrc plugin which is set on command line to point to source audio file. osssink is a consumer only plugin, being at the end of the pipeline, while filesrc is a producer only plugin. Similarly here is a ogg/vorbis player:

The main difference is that ogg-vorbis has a container (the Ogg format) around audio content (the Vorbis stream), so the pipeline needs to unwrap Vorbis data before decoding it; that's done by the couple oggdemux ! vorbisdec, an Ogg demuxer unwrapping content later fed to a Vorbis decoder.

That's both simple and complex at the same time. How I'm supposed to know which plugin connect to which other in forming a pipeline? The answer is very easy. GStreamer plugins provide self inspection. With command line tool gst-inspect you can learn how a plugin interfaces to outside world and what can be connected to it, and also which formats the plugin can receive and produce and a lot more. Calling gst-inspect without parameters will return a full list of all the available plugins. Providing the name of a plugin will return more informations on that plugin only. Let see what can we learn about the first plugin, filesrc:

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      ANY

That section informs us that any kind of output can be produced by this plugin (Capabilities: ANY). That makes sense, since a file sourcing plugin can read any kind of data from inside the file.

Pads:
  SRC: 'src'
    Implementation:
      Has getrangefunc(): gst_base_src_pad_get_range
      Has custom eventfunc(): gst_base_src_event_handler
      Has custom queryfunc(): gst_base_src_query
    Pad Template: 'src'

The plugin provides a pad called src which refers to previously seen 'src' template. So we now know that using this plugin will provides us a src pad which is a producer pad outputting any kind of data in the known and unknown universe. The output is actually much more longer but we don't manage to analyze it all here. That was just to give you a taste of GStreamer plugins. But, as we mentioned before, building a pipeline is quite complex. Another thing to consider is that pads can be static or dynamic. Some file contains an arbitrary number of streams of the same kind (think to multilanguage audio tracks inside an AVI or Ogg file). So certain plugins will create pads dynamically. If you really need all that complexity you can get your hands dirt, but otherwise there is another solution: the playbin pipeline we already mentioned before.

Playbin is a full, dynamic pipeline implemented in a single plugin. To use it you have just to set one or more properties and all the black magic will be done behind the scenes by GStreamer. The most important one is the uri property which points to media to be played, using  a compliant URI syntax, like file:///home/myself/Video/concert.ogg or http://www.on.line.source.net/song.mp3. Doing it on the command line is just a matter of:

gst-launch playbin uri="file:///usr/share/example-content/Experience ubuntu.ogg"

Just issue this command and you'll see and hear your media played for you. We'll use playbin for our movie player in later posts.

I hope the load of information has not been too high. GStreamer is quite easy to be used as a multimedia framework, but quite easy should be intended in relation to the intrinsics complexity of multimedia frameworks. You need to study for some days the good documentation the project provides and you'll be able to start coding more faster than you imagine.

So far, that's all. In the next issue we'll use GStreamer playbin inside our application to provide playing capability. See you next time.