Streaming basics

As a quick crash course on video and audio streaming, here’s some nice facts and articles.

The tool of the trade is RTSP, like it or not. There are methods of streaming through HTTP, but to make something that supports multiple streams or any sort of negotiation, it’ll always come back to RTSP.
RFC 2326 – Real Time Streaming Protocol

Now, in order to encapsulate content and provide it through RTSP, there’s RTP. Yes, the acronyms are very confusing. But remember that RTSP and RTP go hand in hand for most cases.
RFC 3550 – Real-time Transport Protocol

RTP encapsulates any stream protocol that is for the actual media. MPEG-4 streams have their own, H.264 streams their own.
RFC 3016 – RTP payload for MPEG-4 Audio/Visual streams
RFC 3984 – RTP payload for H.264 Video

Now, you can use these RFC documents to make your own RTSP/RTP server or client. Or you can use one of the available libraries.

Libavcodec – can decode most payloads. Note, this is not to be used with the RTP packets themselves.
Libavformat – can handle connections to RTSP servers and read RTP packets and provide the payloads contained. Also it can provide a decodable frame for libavcodec.
Live555 – can handle RTSP connections and read RTP packets.
VLC – uses libavcodec and libavformat for most functions but the source code still provides a lot of insight into handling streams.