Clustering RabbitMQ on ubuntu 18.x

I am exploring a highly available, multi site, multi network rabbitmq server for what it does best.

3 VMs

ub1

ub2 

ub3

on KVM+kimchi virtualization setup.

sudo vim /etc/hosts

and update it with the details of all 3 nodes

192.x.y.z ub1
192.x.y.z ub2
192.x.y.z ub3

Install and Setup RabbitMQ on all

Copy paste the following on all 3 nodes.

sudo apt update
sudo apt upgrade
sudo apt install rabbitmq-server -y

sudo systemctl start rabbitmq-server
sudo systemctl enable rabbitmq-server

Plugins

sudo rabbitmq-plugins enable rabbitmq_management
sudo systemctl restart rabbitmq-server

Firewall

sudo ufw allow ssh
sudo ufw enable
sudo ufw allow 5672,15672,4369,25672/tcp
sudo ufw status

Clustering

So the erlang cookie used by rabbitmq should be same on all hosts/nodes. We can copy it from node1 to others via scp but we need to change the permissions on other nodes for this cookie so that you don't get permission denied error. Run the below command on ub2 & ub3.

sudo chmod 777 /var/lib/rabbitmq/.erlang.cookie

Run the following on ub1

scp /var/lib/rabbitmq/.erlang.cookie root@ub2:/var/lib/rabbitmq/
scp /var/lib/rabbitmq/.erlang.cookie root@ub3:/var/lib/rabbitmq/

Change back the permission on the cookie to be accessible only by the owner. Run the following on ub2,ub3.

sudo chmod 600 /var/lib/rabbitmq/.erlang.cookie

Again on ub2,ub3

sudo systemctl restart rabbitmq-server
sudo rabbitmqctl stop_app

sudo rabbitmqctl join_cluster rabbit@ub1
sudo rabbitmqctl start_app

User Setup

Run the following on ub1. This will create an admin user and deletes the guest user.

sudo rabbitmqctl add_user admin admin
sudo rabbitmqctl set_user_tags admin administrator
sudo rabbitmqctl set_permissions -p / admin ".*" ".*" ".*"
sudo rabbitmqctl delete_user guest

Run the following on ub2,ub3 to confirm that the clustering is working.

sudo rabbitmqctl cluster_status
sudo rabbitmqctl list_users

HA[Queue Mirroring]

sudo rabbitmqctl set_policy ha-all "." '{"ha-mode":"all"}'