Mediastreamer2 - the multimedia streaming engine
Mediastreamer2 is a powerful and lightweighted streaming engine specialized for voice/video telephony applications.
It is the library that is responsible for all the receiving and sending of multimedia streams in linphone, including voice/video capture, encoding and decoding, and rendering.
- Capture and playback from various platform dependent sound architectures (ALSA, AudioUnits, AudioQueue, WaveApi)
- Send and receive RTP streams
- Encode and decode the following audio formats: speex, G711, GSM, iLBC, AMR, AMR-WB, G722, SILK, G729, and video formats H263, theora, MPEG4, H264 and VP8.
- Read and write from/to a wav file
- Read YUV pictures from a camera based on platform dependents APIs.
- Optimized rendering of YUV pictures
- Dual tones generation
- Custom tone detector
- Echo cancelation, based on speex library or webrtc AEC on android
- Audio conferencing
- Audio parametric equalizer
- Volume control, automatic gain control
- ICE for optimized NAT traversal
- Adaptive bitrate control algorithm: for automatic adaption of encoder bitrate based on received RTCP feedback.
Mediastreamer2 can be extended with dynamic plugins, currently H264, ILBC, SILK, AMR, AMR-WB and G729 codec plugins are available.
- Google Android >= 2.2
- Windows XP, Vista and 7
- Mac OS X
- Embedded Linux: ARM and Blackfin
Design and principles
Each processing entity is contained within a MSFilter object. MSFilter(s) have inputs and/or outputs that can be used to connect from and to other MSFilters.
A trivial example to understand:
- MSRtpRecv is a MSFilter that receives RTP packets from the network, unpacketize them and post them on its only output.
- MSSpeexDec is a MSFilter that takes everything on its input assuming these are speex encoded packets, and decodes them and put the result on its output.
- MSFileRec is a MSFilter that takes everything on its input and write it to wav file (assuming the input is 16bit linear pcm).
MSFilters can be connected together to become filter chain. If we assemble the three above examples, we obtain a processing chain that receives RTP packet, decode them and write the uncompressed result into a wav file.
MSRtpRecv --> MSSpeexDec --> MSFileRec
The execution of the media processing work is scheduled by a MSTicker object, a thread that wakes up every 10 ms to process data in all the MSFilter chains it manages. Several MSTicker can be used simultaneously, for example one for audio filters, one for video filters, or one on each processor of the machine where it runs.
Mediastreamer2 is easy to use
If your intent is simply to create audio and video streams, a simple API is defined in mediastreamer2/mediastream.h to create and manage audio and video streams.
If your intent is to add new functionalities to mediastreamer2, you'll be glad to know that implementing a mediastreamer2 filter is very straightforward. The mediastreamer2 filter encapsulation is very light.
Thanks to this lightweighted framework, developers can concentrate on what matters: the implementation of the signal/image processing algorithm !
Mediastreamer2 is documented using doxygen. You can browse the API documention here.
Mediastreamer2 is suitable for embedded systems
- Mediastreamer2 is light. For example on linux/x86 the full-featured shared library takes around 800ko unstripped and compiled with -g (debug). Data messages that carries the media data within mediastreamer2 chains are optimized using the famous sys-V mblk_t structure. This is to avoid copies as long it is possible and allow low cost fragmentation/re-assemble operations that are very common especially when processing video streams.
- Mediastreamer2 is written in C
- Mediastreamer2 compiles on arm with gcc.
- Mediastreamer2 has only oRTP and libc as minimal dependencies. Others (ffmpeg, speex, alsa...) can be added optionnaly if you need all features.
- Thanks to its plugin architecture, mediastreamer2 can be extended to interface with hardware codecs, for example video codecs dsp.
Mediastreamer2 releases can be downloaded here.
You can also use git to retrieve the lastest source code (recommended for developers):
git clone git://git.linphone.org/mediastreamer2.git
Mediastreamer2 is licensed under GPL.
Proprietary licenses can be ordered from Belledonne Communications.
Need commercial support for mediastreamer2 ?