You may want to learn more about Origin Application and Edge Application.
Basic Cluster Configuration
In this guide, we will show you how to configure a basic Storm Streaming Server cluster. To create a cluster, you actually need only one server instance, as it can include the ClusterManager, an Origin application, and an Edge application. This configuration can be an excellent starting point for expanding our infrastructure, where adding additional instances will be seamless.
However, in this guide, we will use 3 instances to better illustrate the division of roles and tasks of each component.
Basic Configuration
In our example configuration, we assume having 3 Storm Streaming Server instances. The first one (IP: 192.168.0.2) is an Origin server and will also have the Cluster Manager active. The other two servers will have Edge-type applications defined (IP: 192.168.0.3 and 192.168.0.4).
ClusterManager (192.168.0.2)Typically, the ClusterManager controller is not enabled; to activate it, we must edit the preferences.xml file in the config directory.
Sample configuration:
<ClusterManager enabled="true">
<Identity>Mother</Identity>
<IPWhiteList></IPWhiteList>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
Explanation:
| Identity | Identifier for the mother, so it is possible to recognize which instance we are dealing with. We recommend using naming like "Primary", "Secondary". |
| IPWhiteList | Access to the ClusterManager can be restricted to a list of defined IP addresses. If this field is empty, no restrictions apply. If we want to define an IP list that the server checks before each connection, simply enter the IP addresses separated by commas, e.g., "192.168.0.2, 192.168.0.3" (whitespace doesn't matter). |
| Secret | This is the password that all applications logging into the Cluster Manager must present. |
Table 1. ClusterManager field explanation table
Origin Applications (192.168.0.2)
Below is an example configuration for an Origin-type application. The most important block here is StreamingClusterSettings, which defines the connection with the ClusterManager. It is possible to define more than one connection, but each must be assigned a priority (higher the number, higher the priority).
Sample configuration:
<Application name="origin" type="origin">
<StreamingClusterSettings>
<SupervisorList>
<ClusterManager host="192.168.0.2" port="8080" isSSL="true">
<Priority>1</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
<PeerAccessSettings host="192.168.0.2" port="1935" isSSL="false"/>
</SupervisorList>
<CopyOnPublish>true</CopyOnPublish>
</StreamingClusterSettings>
...
</Application>
| ClusterManager:host | Host or IP address where the ClusterManager controller is located. |
| ClusterManager:port | Port on which the WebSocket protocol of the ClusterManager controller is available (usually 8080 or 443). |
| ClusterManager:isSS | Whether to use SSL communication with the ClusterManager. |
| Priority | Importance of this connection compared to parallel connections. |
| Secret | Password with which the application will log into the ClusterManager (must be the same). |
| PeerAccessSettings:host | Public host or IP address of this application, so other applications know where to connect. |
| PeerAccessSettings:port | Public port of this application, so other applications know where to connect (indicate a port with an active RTMP protocol). |
| PeerAccessSettings:isSSL | Whether the connection should use SSL. |
Table 2. Origin/Edge field explanation table
You can learn more about specifics configurations in: Origin Application Guide.
Edge Applications (192.168.0.3 & 192.168.0.4)
Below is an example configuration for an Edge-type application - it's very similar to Origin application with just a few additional parameters.
Sample configuration:
<Application name="edge" type="edge">
<StreamingClusterSettings>
<SupervisorList>
<ClusterManager host="192.168.0.2" port="8080" isSSL="false">
<Priority>1</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
</SupervisorList>
<CopyOnPublish>true</CopyOnPublish>
</StreamingClusterSettings>
...
</Application>
| ClusterManager:host | Host or IP address where the ClusterManager controller is located. |
| ClusterManager:port | Port on which the WebSocket protocol of the ClusterManager controller is available (usually 8080 or 443). |
| ClusterManager:isSS | Whether to use SSL communication with the ClusterManager. |
| Priority | Importance of this connection compared to parallel connections. |
| Secret | Password with which the application will log into the ClusterManager (must be the same). |
| CopyOnPublish | Whenever stream from an Origin application should be immediately copied to this Edge application, otherwise it'll wait for a first request from a viewer. |
Table 3. Origin/Edge field explanation table
You can learn more about specifics configurations in: Edge Application Guide.
Redundancy
As mentioned earlier, a single server instance can only support a single ClusterManager controller. However, for each Edge and Origin application, it is possible to define connections with multiple such controllers located on different servers. This ensures that in case of a failure of the main instance, the cluster will continue to operate without interruption. When the main ClusterManager regains functionality, all applications will reconnect to it.
Sample configuration:
<Application name="edge" type="edge">
<StreamingClusterSettings>
<SupervisorList>
<ClusterManager host="192.168.0.2" port="8080" isSSL="false">
<Priority>2</Priority>
<Secret>qwerty*1234556</Secret>
</ClusterManager>
<ClusterManager host="192.168.0.10" port="8080" isSSL="false">
<Priority>1</Priority>
<Secret>1234567qwe</Secret>
</ClusterManager>
</SupervisorList>
<CopyOnPublish>true</CopyOnPublish>
</StreamingClusterSettings>
...
</Application>
Support Needed?
Create a free ticket and our support team will provide you necessary assistance.