Gstreamer: an Overview
10 likes3,983 views
GStreamer is a multimedia framework for building applications that handle audio and video content. It provides reusable code modules called elements that can be linked together into pipelines to process multimedia. Elements include sources, filters, decoders, encoders and sinks. Pads connect elements and negotiate compatible data formats. Pipelines synchronize playback across elements and the framework supports integration with other projects and bindings to multiple programming languages.
Gstreamer: an Overview
- 1. GStreamer: An overview Rodrigo Costa Pontifical Catholic University of Rio de Janeiro TeleM´ıdia Lab. rodrigocosta@telemidia.puc-rio.br April 16, 2015 Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 1 / 22
- 2. Overview 1 GStreamer Overview Pipelines Basic Concepts Synchronization Examples Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 2 / 22
- 3. What is GStreamer? A framework for multimedia applications Linux, Windows, OS X, . . . 10+ years old project Similar to DirectShow, QuickTime, etc. Designed to make it easy to write applications that handle audiovisual contents Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 3 / 22
- 4. GStreamer overview What GStreamer NOT is? A codec implementation An application itself Features Fully extensible by plugins Flexible Low and high level API Often wraps other libraries (decoders, encoders, filters, etc.) Integration with other frameworks and projects WebKit, Firefox, Gnome, Android, iOS, . . . Pipeline-based Bindings to multiple languages C/C++, Python, Java, Ruby, Pearl, etc. Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 4 / 22
- 5. GStreamer overview Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 5 / 22
- 6. Design The fundamental design comes from the video pipeline at Oregon Graduate Institute: Infopipes Infopipes (2001) High-level abstraction for building blocks that handle information flows Goal: simplify the task of building distributed streaming applications by providing basic elements such as pipes, filters, buffers, and pumps Motivated by the lack of support of RPC-like technologies to develop streaming and processing of distributed information Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 6 / 22
- 7. Infopipes Infopipe example: Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 7 / 22
- 8. GStreamer pipeline GStreamer pipeline example: Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 8 / 22
- 9. Basic Concepts Elements The most important class of objects in GStreamer Each one has a specific function: read a file, receive data from network, decode, multiplex, present the data, . . . A chain of elements is created and linked together for the data to flow through it GStreamer ships with a large set of elements Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 9 / 22
- 10. Basic Concepts Elements (cont) Source Filters, converters, demuxers, muxers, codecs Sink Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 10 / 22
- 11. Basic Concepts Pads Connection points between elements Source pads produce data Sink pads consume data Data always flows from src to sink pads Can operate in pull or push mode Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 11 / 22
- 12. Basic Concepts Pads (cont) Each pad have a predefined set of properties called Capabilities or Caps Caps are used to validate the communication between elements (caps restrict the type of data that flows through it) A source pad can only be linked to a sink pad if their allowed data types are compatible For instance, a source pad that produces a video stream cannot be linked to a sink pad that consumes audio streams Elements can negotiate with each other on format to use (caps negotiation) Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 12 / 22
- 13. Basic Concepts Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 13 / 22
- 14. Basic Concepts Bins Elements can be grouped into a container called bin It can be manipulated as it were an element It also manages the state of its children elements Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 14 / 22
- 15. Basic Concepts Pipeline A pipeline is a top-level bin It provides a bus for communication purposes It manages the playout synchronization It runs in a separate thread Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 15 / 22
- 16. Basic Concepts Communication: Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 16 / 22
- 17. Synchronization GstClock objects provide clock time GstClock always returns the absolute-time The pipeline maintains a GstClock object and a base-time (time when it goes to PLAYING state) running-time = absolute-time - base-time Sink elements are responsible for present the buffers in their respective presentation time If a buffer is delayed, the sink drops it Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 17 / 22
- 18. Pipeline Example Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 18 / 22
- 19. Example 1 Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 19 / 22
- 20. Example 2 Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 20 / 22
- 21. Example 3 Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 21 / 22
- 22. Example 4 Rodrigo Costa (TeleM´ıdia Lab.) GStreamer April 16, 2015 22 / 22