Meet the RTMP (Real-Time Messaging Protocol)

For everyone who is just beginning their journey with creating video streaming services, understanding and becoming familiar with the currently used streaming protocols is essential. In this comprehensive article, I will allow myself to describe them all, focusing on their disadvantages, advantages, and applications.

RTMP History

Let’s perhaps start from the history of this protocol, which from its very beginning was inextricably linked with the popular Macromedia Flash Player plug-in (later known as Adobe Flash Player). The plug-in itself was revolutionary in many aspects – it allowed the creation of complex animations and even fully-fledged games within a browser environment. The scripting language known as ActionScript gave creators vast possibilities, which far surpassed what JavaScript offered at the time (like Object-Oriented programming). One of the revolutions initiated by the Flash plug-in was video on the Internet (the first youtube.com video player was written in this technology).

The next step was to enable the creation of real-time streaming using the proprietary RTMP protocol and the dedicated Flash Media Server software. This was around year 2002. As of 2024, Adobe Flash Player has been consigned to oblivion, but the RTMP protocol remains with us to this day, and there are no signs indicating its imminent demise. To this day, it remains the most popular option for broadcasting live streams to services such as YouTube, Twitch, Facebook…

Usage and main characteristics

Originally, the RTMP protocol was used directly in Flash applications for both receiving video broadcasts and streaming them. With the demise of the plug-in itself, the role of the protocol has also changed. To this day, it has remained the fundamental choice for broadcasting video and sound to streaming servers by content creators (the process is called “ingest”). Programs such as OpenBroadcaster, XSplit, Streamlabs utilize RTMP as the primary and default protocol for connections with streaming servers. And although there are many different protocols that fulfill a role like RTMP, it still enjoys great popularity.

Advantages

The main features of the RTMP protocol include its relative speed, as it is based on TCP/IP (there have been versions designed for UDP as well), which enables low latency of about 0.5-0.7 seconds. It is suitable for both downloading the stream by the viewer (PLAY function) and publishing it (PUBLISH function).

Other advantages of the RTMP protocol include:

• The capability to transfer data alongside video (for example, by serializing data into AMF1/3 format).
• Dynamic frame size management (adjusting how much data is transferred at once between the client and server) to maximize efficiency.
• Support for a system of “states” that indicate whether a given stream is ready or if its broadcast has not yet started.
• A mechanism for stream publish authorization (the server can require the provision of a proper username and password).
• RTMP allows for seek functionality, which works a tiny bit like nDVR.
• It remains one of the most popular data protocols, utilized by many online services to this day.

Disadvantages

RTMP has two fundamental drawbacks that one must be aware of. Firstly, no web browser natively supports this protocol, meaning there’s no possibility to simply play such a stream directly. Consequently, the role of the protocol has been limited to “delivering” the stream to the origin servers of online services.

The second significant drawback relates to supported codecs. The RTMP protocol itself supports a rather extensive list of video codecs such as: Sorenson Spark, H.264. In the case of audio, it supports for examle MP3, and AAC. However, nowadays, only H.264 and AAC are in use. The other codecs, from today’s perspective, are virtually archaic and are really not applied anywhere.

Other disadvantages of the RTMP protocol include:

• Lack of a Digital Rights Management (DRM) mechanism, which would protect video content from unauthorized copying,
• No auto-reconnect mechanism in case of connection loss (implementation is required on the client-side).
• No support for multiple tracks per video stream (Adaptive Bitrate)

Supported codecs

As mentioned above, the list of codecs supported by the RTMP protocol is quite extensive, but most of them are very outdated and not practically used. The situation is remedied by an improved version of the protocol named Enhanced RTMP.

Supported video codecs:

• Sorenson Spark (H.263 implementation)
• AVC / H.264
• Screen Video v1
• Screen Video v2
• On2 along with a version supporting the alpha channel
• HEVC/H.265 (only in Enhanced RTMP)
• AV1 (only in Enhanced RTMP)
• VP8 (only in Enhanced RTMP)
• VP9 (only in Enhanced RTMP)

Supported audio codecs:

• AAC
• AAC-3 (only in Enhanced RTMP)
• E-AC-3 (only in Enhanced RTMP)
• MP3
• Speex

Enhanced RTMP

In 2022, a non-profit organization named Veovera Software Organization was established with the goal of developing an updated version of the protocol called “Enhanced RTMP.” The basic principles of the project include the integration of new codecs such as VP8, VP9, H.265 (HEVC), and AV1. Additionally, the project aims to add support for multiple tracks (Multitrack) and features such as a reconnect request, all while maintaining backward compatibility with the base protocol. The project has a repository on GitHub where progress on the development of further changes can be tracked: https://github.com/veovera/enhanced-rtmp).

Protocol versions

There are several versions of the RTMP protocol, which differ in the encryption mechanisms used or the underlying protocol employed. The vast majority of them are no longer commonly used, however, a significant number of encoders, such as FFMPEG, are capable of handling them. It’s worth noting that Enhanced RTMP is not considered a separate protocol in this context and theoretically can be applied with any encryption or tunneling mechanism.

• RTMP – the base protocol, which does not have any encryption mechanism. It commonly uses port 1935.

• RTMPS – the RTMP protocol with added SSL/TLS support. When streaming to popular services like YouTube or Facebook, RTMPS is the protocol of choice. Default port is usually 443.

• RTMPE – an obsolete version of the RTMP protocol with a proprietary encryption mechanism developed by Adobe.

• RTMPT – a variant of the protocol that uses HTTP as its carrier. This approach allows it to bypass many firewalls through data encapsulation. Unfortunately, the downside of this solution is relatively large delays compared to its base version. Default ports are usually 80 or 443 for SSL encryption.

• RTMPFP – the base RTMP protocol is based on the TCP-IP mechanism, which ensures data integrity and proper sequencing. On the other hand, RTMPFP is based on the UDP protocol, which in theory allows for reduced latency.

Popular encoders supporting RTMP

List of popular software encoders used for streaming with RTMP protocol.

• OBS (Open Broadcaster Software) is a popular open-source streaming and recording program widely used for live streaming on platforms like Twitch and YouTube. It supports RTMP streaming directly, allowing users to customize their stream’s resolution, bitrate, and other settings easily. OBS’s versatility and plugin support make it a preferred choice for both beginners and professionals.

• XSplit offers both Broadcaster and Gamecaster versions, tailored for professional broadcasters and gamers respectively. The software supports RTMP streaming, providing features like scene transitions, source transitions, and pre-loaded plugins to enhance the streaming experience. XSplit is known for its intuitive interface, which simplifies the setup process for streaming via RTMP.

• Wirecast is a professional live video streaming production tool by Telestream, designed for serious broadcasters. It supports RTMP streaming and includes features like multi-camera switching, overlays, lower thirds, and other professional attributes. Wirecast is ideal for producing high-quality live streams and is used extensively in educational and professional settings.

• Streamlabs is a powerful streaming software that integrates well with streaming platforms to utilize features like alerts, donations, and chat. It supports RTMP streaming, enabling users to broadcast live to any RTMP-enabled platform seamlessly. Streamlabs is favored for its user-friendly interface and rich integration with other services.

• FFmpeg is a comprehensive, cross-platform solution to record, convert, and stream audio and video. It includes support for RTMP streaming through its command-line interface, which allows for highly customizable streaming setups. Although it requires more technical knowledge, FFmpeg is incredibly powerful for developers and technical users looking for a customizable streaming solution.

• vMix is a live production software that turns your computer into a production studio for live video streaming. It supports RTMP streaming and offers features like live mixing, switching, and streaming of SD, full HD, and 4K video sources including cameras, video files, DVDs, images, and more. vMix is well-suited for producing professional-quality broadcasts on a budget.

Protocol mechanics

RTMP protocol was designed for high-performance transmission of audio, video, and data between a server and a client. Here’s an in-depth look at how RTMP works from network and packets perspective, starting with the handshake process and moving through packet exchanges and command functions, concluding with data encapsulation formats AMF0 and AMF3.

RTMP Handshake

The RTMP handshake is crucial for establishing a connection between the client and server. It consists of three parts: C0, C1, and C2 for the client, and S0, S1, and S2 for the server.

C0 and S0: These are the first packets exchanged in the RTMP handshake. Both C0 (Client) and S0 (Server) contain a single byte, representing the protocol version (typically 0x03 for RTMP).

C1 and S1: Following C0 and S0, both the client and server send their respective C1 and S1 packets. These packets are 1536 bytes each and contain the timestamp and a random echo field. The timestamp in C1 is the current time of the client when the packet is created, and S1 includes the server’s timestamp.

C2 and S2: C2 is sent by the client and mirrors the timestamp and the random data received from S1. Similarly, S2 is sent by the server, mirroring the timestamp and random data received from C1. This step ensures both the server and client have received each other’s messages correctly and are synchronized.

RTMP Packet Exchange

After the handshake, the client and server communicate via RTMP packets, which are categorized into several types including command messages, data messages, shared object messages, audio messages, video messages, and user control messages.

RTMP Commands

RTMP commands are transmitted in command messages, typically encapsulated using AMF0 or AMF3 formats. Some of the fundamental commands include:

• Connect: Initiates a connection between the client and the RTMP server.
• Publish: Used by a client to start publishing a stream. It includes the stream name and type (live, record, or append).
• Play: Sent by the client to play a stream from the server. The client specifies the stream name and other optional parameters like start time and duration.
• CreateStream: Requested by the client to instruct the server to create a new stream identified by a unique stream ID.

Data Types: AMF0 and AMF3

Adobe Message Format (AMF) is used for serializing data in RTMP. There are two versions used within RTMP:

AMF0: The original version used for encoding and decoding the data in command messages and other parts of the RTMP. It supports data types like numbers, booleans, strings, objects, null, undefined, references, ECMA arrays, object end markers, strict arrays, and dates.

AMF3: Introduced with the advent of ActionScript 3 and Flex 2, AMF3 provides a more compact serialization format for complex data types. It is typically used alongside AMF0; for example, AMF0 is used for initial commands in the handshake and setting up the connection, while AMF3 may be used for actual data transfer.

Summary

In the evolving landscape of digital streaming, RTMP (Real-Time Messaging Protocol) stands as a stalwart, demonstrating remarkable resilience and adaptability over the years. As we reflect on the protocol’s journey and its ongoing relevance, it’s important to recognize both its legacy and its potential future within the industry.

Initially developed for seamless integration with Macromedia Flash Player, RTMP has transcended its origins to become a cornerstone of live broadcasting on platforms like YouTube, Twitch, and Facebook. This enduring popularity is rooted in the protocol’s robust feature set, which caters adeptly to the dynamic needs of real-time media delivery.

Storm Streaming Server & Cloud Support for RTMP

Both Storm Streaming Server and Storm Streaming Cloud fully support the RTMP protocol, including the Enhanced version, allowing streaming with the latest codecs such as H.265 and AV1. When configuring our server application, we can decide whether to require authorization to publish the stream, or even to play it back. In addition to the standard function of accepting a stream (ingest), Storm Streaming Server can also copy an external RTMP stream (pull) and push out such a stream (push).