RabbitMQ installation guide on Linux Ubuntu 16.04
Introduction
With more than 35,000 production deployments of RabbitMQ world-wide at small startups and large enterprises, RabbitMQ is the most popular open source message broker.
RabbitMQ is lightweight and easy to deploy on premise and in the cloud. It supports multiple messaging protocols. RabbitMQ can be deployed in distributed and federated configurations to meet high-scale, high-availability requirements.
In this guide, we’ll cover how to install and configure a 3-node rabbitMQ cluster, with HA proxy and management plugin.
Installing RabbitMQ on single node
Update our system’s default application toolset
$ sudo apt-get update $ sudo apt-get -y upgrade
Enable RabbitMQ appication repository
$ echo "deb http://www.rabbitmq.com/debian/ testing main" >> /etc/apt/sources.list
Add the verification key for the package
$ curl http://www.rabbitmq.com/rabbitmq-signing-key-public.asc | sudo apt-key add -
Update the source with our new addition from above
$ sudo apt-get update
And finally, download and install RabbitMQ
$ sudo apt-get install rabbitmq-server
Start rabbitMQ server
$ sudo systemctl restart rabbitmq-server.service
Congratulation!! You’ve just successfully install your rabbitMQ server.
Configure multi-cluster RabbitMQ
In this part, we’ll go through how to setup a multi-node RabbitMQ cluster system (3 nodes in this example).
Sync the erlang cookie
RabbitMQ nodes and CLI tools (e.g. rabbitmqctl) use a cookie to determine whether they are allowed to communicate with each other. For two nodes to be able to communicate they must have the same shared secret called the Erlang cookie. The cookie is just a string of alphanumeric characters. It can be as long or short as you like. Every cluster node must have the same cookie.
Erlang VM will automatically create a random cookie file when the RabbitMQ server starts up. The easiest way to proceed is to allow one node to create the file, and then copy it to all the other nodes in the cluster.
On Unix systems, the cookie will be typically located in /var/lib/rabbitmq/.erlang.cookie or $HOME/.erlang.cookie.
When the cookie is misconfigured (for example, not identical), RabbitMQ will log errors such as “Connection attempt from disallowed node” and “Could not auto-cluster”.
Configure hostname and RabbitMQ Nodename
Configure Hostname
Edit hosts:
$ sudo vim /etc/hosts
xxx.xxx.xxx.xxx rabbit01 xxx.xxx.xxx.xxx rabbit02 xxx.xxx.xxx.xxx rabbit03
Configure RabbitMQ hostname
Edit rabbitmq-evn.conf:
rabbit01$ sudo vim /etc/rabbitmq/rabbitmq-env.conf NODENAME=rabbit@rabbit01 rabbit02$ sudo vim /etc/rabbitmq/rabbitmq-env.conf NODENAME=rabbit@rabbit02 rabbit03$ sudo vim /etc/rabbitmq/rabbitmq-env.conf NODENAME=rabbit@rabbit03
Staring independent nodes
Clusters are set up by re-configuring existing RabbitMQ nodes into a cluster configuration. Hence the first step is to start RabbitMQ on all nodes in the normal way:
rabbit01$ rabbitmq-server -detached rabbit02$ rabbitmq-server -detached rabbit03$ rabbitmq-server -detached
This creates three independent RabbitMQ brokers, one on each node, as confirmed by the cluster_status command:
rabbit01$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit01 ...
[{nodes,[{disc,[rabbit@rabbit01]}]},{running_nodes,[rabbit@rabbit01]}]
...done.
rabbit02$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit02 ...
[{nodes,[{disc,[rabbit@rabbit02]}]},{running_nodes,[rabbit@rabbit02]}]
...done.
rabbit03$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit03 ...
[{nodes,[{disc,[rabbit@rabbit03]}]},{running_nodes,[rabbit@rabbit03]}]
...done.
The node name of a RabbitMQ broker started from the rabbitmq-server shell script is rabbit@shorthostname, where the short node name is lower-case (as in rabbit@rabbit01, above). If you use the rabbitmq-server.bat batch file on Windows, the short node name is upper-case (as in rabbit@RABBIT01). When you type node names, case matters, and these strings must match exactly.
Create the cluster
In order to link up our three nodes in a cluster, we tell two of the nodes, say rabbit@rabbit2 and rabbit@rabbit3, to join the cluster of the third, say rabbit@rabbit1.
We first join rabbit@rabbit2 in a cluster with rabbit@rabbit1. To do that, on rabbit@rabbit2 we stop the RabbitMQ application and join the rabbit@rabbit1 cluster, then restart the RabbitMQ application. Note that joining a cluster implicitly resets the node, thus removing all resources and data that were previously present on that node.
rabbit02$ rabbitmqctl stop_app Stopping node rabbit@rabbit02 ...done. rabbit02$ rabbitmqctl join_cluster rabbit@rabbit01 Clustering node rabbit@rabbit02 with [rabbit@rabbit01] ...done. rabbit02$ rabbitmqctl start_app Starting node rabbit@rabbit02 ...done.
We can see that the two nodes are joined in a cluster by running the cluster_status command on either of the nodes:
rabbit01$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit1 ...
[{nodes,[{disc,[rabbit@rabbit01,rabbit@rabbit02]}]},
{running_nodes,[rabbit@rabbit02,rabbit@rabbit01]}]
...done.
rabbit02$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit02 ...
[{nodes,[{disc,[rabbit@rabbit01,rabbit@rabbit02]}]},
{running_nodes,[rabbit@rabbit01,rabbit@rabbit02]}]
...done.
Now we join rabbit@rabbit03 to the same cluster. The steps are identical to the ones above, except this time we’ll cluster to rabbit02 to demonstrate that the node chosen to cluster to does not matter – it is enough to provide one online node and the node will be clustered to the cluster that the specified node belongs to.
rabbit03$ rabbitmqctl stop_app Stopping node rabbit@rabbit03 ...done. rabbit03$ rabbitmqctl join_cluster rabbit@rabbit02 Clustering node rabbit@rabbit03 with rabbit@rabbit02 ...done. rabbit03$ rabbitmqctl start_app Starting node rabbit@rabbit03 ...done.
We can see that the three nodes are joined in a cluster by running the cluster_status command on any of the nodes:
rabbit01$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit01 ...
[{nodes,[{disc,[rabbit@rabbit01,rabbit@rabbit02,rabbit@rabbit03]}]},
{running_nodes,[rabbit@rabbit03,rabbit@rabbit02,rabbit@rabbit01]}]
...done.
rabbit02$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit02 ...
[{nodes,[{disc,[rabbit@rabbit01,rabbit@rabbit02,rabbit@rabbit03]}]},
{running_nodes,[rabbit@rabbit03,rabbit@rabbit01,rabbit@rabbit02]}]
...done.
rabbit03$ rabbitmqctl cluster_status
Cluster status of node rabbit@rabbit03 ...
[{nodes,[{disc,[rabbit@rabbit03,rabbit@rabbit02,rabbit@rabbit01]}]},
{running_nodes,[rabbit@rabbit02,rabbit@rabbit01,rabbit@rabbit03]}]
...done.
By following the above steps we can add new nodes to the cluster at any time, while the cluster is running.
Management Plugin
Enable rabbitmq_management in everynode
$ sudo rabbitmq-plugins enable rabbitmq_management
Create admin user on one node, this admin user can be use for any node in cluster
$ sudo rabbitmqctl add_user <user> <password> $ sudo rabbitmqctl set_user_tags <user> administrator $ sudo rabbitmqctl set_permissions -p / <user> ".*" ".*" ".*"
Now you can access management website with this user through xxx.xxx.xxx.xxx:15672
Setup HAProxy
Install haproxy
$ sudo apt-get install haproxy
Verify if HAProxy is working
$ haproxy -v HA-Proxy version 1.6.3 2015/12/25 Copyright 2000-2015 Willy Tarreau
Configure HAproxy for RabbitMQ and RabbitMQ_management
$ sudo vim /etc/haproxy/haproxy.cfg
defaults
...
listen rabbitmq
bind 0.0.0.0:5672
mode tcp
balance roundrobin
timeout client 3h
timeout server 3h
option clitcpka
server rabbit01 xxx.xxx.xxx.xx1:5672 check inter 5s fall 3 rise 2
server rabbit02 xxx.xxx.xxx.xx2:5672 check inter 5s fall 3 rise 2
server rabbit02 xxx.xxx.xxx.xx3:5672 check inter 5s fall 3 rise 2
listen rabbitmq_management
bind 0.0.0.0:15672
mode tcp
balance roundrobin
server rabbit01 xxx.xxx.xxx.xx1:15672 check fall 3 rise 2
server rabbit02 xxx.xxx.xxx.xx2:15672 check fall 3 rise 2
server rabbit02 xxx.xxx.xxx.xx3:15672 check fall 3 rise 2