Clusters & Scaling - Storm Streaming Server

The Storm streaming server was designed from the beginning with scalability and ease of expansion in mind. Scalability not only allows for an increase in the maximum number of viewers for the entire system but also its redundancy in case one or more servers fail.

Introduction

The Storm Streaming Server cluster consists of two types of components. The first is the ClusterManager controller - to which all other components are connected, and it acts as the brain of the entire structure. Unlike typical "applications", a single server instance can have only one ClusterManager, but more ClusterManager instances can be launched within the cluster for redundancy. The second component is applications. A basic "live" type application can create, transcode, and redistribute any stream. However, Storm Streaming Server also includes specialized applications that share their workload.

Cluster Components

The basic architecture of the streaming cluster consists of the following elements:

ClusterManager	This is a special cluster controller responsible for effective communication between all connected applications. It informs connected Edge-type applications about new streams or changes in their states (e.g., a stream has ended), which it receives from Origin-type applications. A single Storm Streaming Server can have only one ClusterManager instance, but each application (edge/origin) can define more than one connection with it.
Origin-type Application	This is a specialized cluster application that essentially performs two roles - it receives a stream (e.g., from OpenBroadcaster) and transcodes it if configured. Upon any change in the stream's state, it informs one or more ClusterManagers about the fact, which then disseminate the information further to Edge servers. The configuration of the Origin Application itself is very similar to live applications and more can be learned in the Origin Application configuration section.
Edge-type Application	Edge-type Application - The role of this application is to distribute the stream to the end user, the viewer. This application draws information from the ClusterManager about where individual streams are located, and then replicates them. Any change in the stream state on the Origin server is automatically reflected on the Edge server. An Edge-type application cannot receive streams nor can it transcode them. Detailed guide for Edge can be found in Origin Application configuration section.

Table 1. Cluster Components table

Basic Configuration

In our example configuration, we assume having 3 Storm Streaming Server instances. The first one (IP: 192.168.0.2) is an Origin server and will also have the Cluster Manager active. The other two servers will have Edge-type applications defined (IP: 192.168.0.3 and 192.168.0.4).

ClusterManager (192.168.0.2)

Typically, the ClusterManager controller is not enabled; to activate it, we must edit the preferences.xml file in the config directory.

Sample configuration:

                        
<ClusterManager enabled="true">
    <Identity>Mother</Identity>
    <IPWhiteList></IPWhiteList>
    <Secret>qwerty*1234556</Secret>
</ClusterManager>

Identity	Identifier for the mother, so it is possible to recognize which instance we are dealing with. We recommend using naming like "Primary", "Secondary".
IPWhiteList	Access to the ClusterManager can be restricted to a list of defined IP addresses. If this field is empty, no restrictions apply. If we want to define an IP list that the server checks before each connection, simply enter the IP addresses separated by commas, e.g., "192.168.0.2, 192.168.0.3" (whitespace doesn't matter).
Secret	this is the password that all applications logging into the Cluster Manager must present.

Table 1. ClusterManager field explanation table

Origin Applications (192.168.0.2)

Below is an example configuration for an Origin-type application. The most important block here is StreamingClusterSettings, which defines the connection with the ClusterManager. It is possible to define more than one connection, but each must be assigned a priority (higher the number, higher the priority).

Sample configuration:

                        
<Application name="origin" type="origin">
    <StreamingClusterSettings>
        <SupervisorList>
            <ClusterManager host="192.168.0.2" port="8080" isSSL="true">
                <Priority>1</Priority>
                <Secret>qwerty*1234556</Secret>
            </ClusterManager>
            <PeerAccessSettings host="192.168.0.2" port="1935" isSSL="false"/>
        </SupervisorList>
    </StreamingClusterSettings>
    ...
</Application>

ClusterManager:host	Host or IP address where the ClusterManager controller is located.
ClusterManager:port	Port on which the WebSocket protocol of the ClusterManager controller is available (usually 8080 or 443).
ClusterManager:isSS	Whether to use SSL communication with the ClusterManager.
Priority	Importance of this connection compared to parallel connections.
Secret	Password with which the application will log into the ClusterManager (must be the same).
PeerAccessSettings:host	Public host or IP address of this application, so other applications know where to connect.
PeerAccessSettings:port	Public port of this application, so other applications know where to connect (indicate a port with an active RTMP protocol).
PeerAccessSettings:isSSL	Whether the connection should use SSL.

Table 1. Origin/Edge field explanation table

You can learn more about specifics configurations in: Origin Application Guide.

Edge Applications (192.168.0.3 & 192.168.0.4)

Below is an example configuration for an Edge-type application - it's very similar to Origin application with just a few additional parameters.

Sample configuration:

                        
<Application name="edge" type="edge">
    <StreamingClusterSettings>
        <SupervisorList>
            <ClusterManager host="192.168.0.2" port="8080" isSSL="false">
                <Priority>1</Priority>
                <Secret>qwerty*1234556</Secret>
            </ClusterManager>
        </SupervisorList>
        <CopyOnPublish>true</CopyOnPublish>
    </StreamingClusterSettings>
    ...
</Application>

ClusterManager:host	Host or IP address where the ClusterManager controller is located.
ClusterManager:port	Port on which the WebSocket protocol of the ClusterManager controller is available (usually 8080 or 443).
ClusterManager:isSS	Whether to use SSL communication with the ClusterManager.
Priority	Importance of this connection compared to parallel connections.
Secret	Password with which the application will log into the ClusterManager (must be the same).
CopyOnPublish	Whenever stream from an Origin application should be immediately copied to this Edge application, otherwise it'll wait for a first request from a viewer.

Table 1. Origin/Edge field explanation table

You can learn more about specifics configurations in: Edge Application Guide.

Redundancy

As mentioned earlier, a single server instance can only support a single ClusterManager controller. However, for each Edge and Origin application, it is possible to define connections with multiple such controllers located on different servers. This ensures that in case of a failure of the main instance, the cluster will continue to operate without interruption. When the main ClusterManager regains functionality, all applications will reconnect to it.

Sample configuration:

                        
<Application name="edge" type="edge">
    <StreamingClusterSettings>
        <SupervisorList>
            <ClusterManager host="192.168.0.2" port="8080" isSSL="false">
                <Priority>2</Priority>
                <Secret>qwerty*1234556</Secret>
            </ClusterManager>
            <ClusterManager host="192.168.0.10" port="8080" isSSL="false">
                <Priority>1</Priority>
                <Secret>1234567qwe</Secret>
            </ClusterManager>
        </SupervisorList>
        <CopyOnPublish>true</CopyOnPublish>
    </StreamingClusterSettings>
    ...
</Application>

Control Panel

A dedicated module devoted to the cluster is also available in the control panel. By navigating to http://.../cpanel/clustermanager, you gain access to a list of applications logged in the cluster (along with detailed information about them) and the currently stored global list of streams. These subpages allow you to quickly verify whether servers are correctly logging into the ClusterManager and whether the streams are indeed being published there.

For more information regarding Storm's Control Panel, please check our Control Panel Guide.

On this page