RabbitMQ HA Cluster

advanced

High-availability RabbitMQ cluster with mirrored queues, management UI, and monitoring.

Overview

RabbitMQ is an open-source message broker that implements the Advanced Message Queuing Protocol (AMQP), originally developed by Rabbit Technologies and later acquired by VMware. It serves as a reliable intermediary for message passing between applications, providing features like message persistence, routing flexibility, and delivery acknowledgments. RabbitMQ supports multiple messaging protocols including AMQP, MQTT, and STOMP, making it versatile for various messaging patterns from simple task queues to complex routing scenarios. This high-availability cluster combines three RabbitMQ nodes with HAProxy load balancing to eliminate single points of failure in message broker infrastructure. The cluster uses Erlang's built-in clustering capabilities with mirrored queues to ensure message durability and availability even when individual nodes fail. HAProxy provides intelligent load balancing for both AMQP connections and management UI access, while Prometheus collects metrics from all components and Grafana visualizes cluster health, message rates, and queue depths. Production environments running microservices, e-commerce platforms, or IoT systems benefit significantly from this stack when message delivery reliability is critical. The combination addresses scenarios where message loss would impact business operations or user experience, providing automatic failover, queue mirroring across nodes, and comprehensive monitoring to detect issues before they affect applications. Organizations transitioning from single RabbitMQ instances or seeking to improve their messaging infrastructure's resilience will find this cluster configuration essential for maintaining service availability.

Key Features

Three-node RabbitMQ cluster with automatic leader election and node synchronization
Mirrored queues across all cluster nodes ensuring message persistence during node failures
HAProxy load balancing for AMQP connections on port 5672 with health checking
Management UI load balancing with sticky sessions for consistent administrative access
Prometheus metrics collection from RabbitMQ nodes including queue depths and message rates
Grafana dashboards for real-time cluster monitoring and historical performance analysis
Erlang cookie-based cluster authentication for secure inter-node communication
Support for multiple messaging protocols (AMQP, MQTT, STOMP) across the cluster

Common Use Cases

1E-commerce platforms requiring guaranteed order processing and inventory updates
2Microservices architectures needing reliable inter-service communication
3IoT data collection systems handling sensor data from thousands of devices
4Financial systems processing payment transactions with zero tolerance for message loss
5Task queue systems distributing background jobs across multiple worker processes
6Event-driven architectures requiring message routing between multiple applications
7Enterprise integration scenarios connecting legacy systems with modern applications

Prerequisites

Minimum 4GB RAM for the complete stack (1GB per RabbitMQ node + 1GB for monitoring)
Docker Engine version 20.10+ with Docker Compose v2 support
Available ports: 5672 (AMQP), 15672 (Management), 8404 (HAProxy stats), 9090 (Prometheus), 3000 (Grafana)
Basic understanding of AMQP concepts and RabbitMQ clustering mechanisms
Familiarity with HAProxy configuration for troubleshooting load balancer issues
Knowledge of PromQL for customizing Prometheus queries and Grafana dashboards

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

1services: 
2  rabbitmq1: 
3    image: rabbitmq:3-management
4    hostname: rabbitmq1
5    environment: 
6      RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
7      RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
8      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
9    volumes: 
10      - rabbitmq1_data:/var/lib/rabbitmq
11      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
12      - ./definitions.json:/etc/rabbitmq/definitions.json:ro
13    networks: 
14      - rabbitmq-net
15    restart: unless-stopped
16
17  rabbitmq2: 
18    image: rabbitmq:3-management
19    hostname: rabbitmq2
20    environment: 
21      RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
22      RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
23      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
24    volumes: 
25      - rabbitmq2_data:/var/lib/rabbitmq
26      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
27    depends_on: 
28      - rabbitmq1
29    networks: 
30      - rabbitmq-net
31    restart: unless-stopped
32
33  rabbitmq3: 
34    image: rabbitmq:3-management
35    hostname: rabbitmq3
36    environment: 
37      RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
38      RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
39      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
40    volumes: 
41      - rabbitmq3_data:/var/lib/rabbitmq
42      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
43    depends_on: 
44      - rabbitmq1
45    networks: 
46      - rabbitmq-net
47    restart: unless-stopped
48
49  haproxy: 
50    image: haproxy:latest
51    ports: 
52      - "5672:5672"
53      - "15672:15672"
54      - "8404:8404"
55    volumes: 
56      - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
57    depends_on: 
58      - rabbitmq1
59      - rabbitmq2
60      - rabbitmq3
61    networks: 
62      - rabbitmq-net
63    restart: unless-stopped
64
65  prometheus: 
66    image: prom/prometheus:latest
67    ports: 
68      - "9090:9090"
69    volumes: 
70      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
71      - prometheus_data:/prometheus
72    networks: 
73      - rabbitmq-net
74    restart: unless-stopped
75
76  grafana: 
77    image: grafana/grafana:latest
78    ports: 
79      - "3000:3000"
80    environment: 
81      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
82    volumes: 
83      - grafana_data:/var/lib/grafana
84    depends_on: 
85      - prometheus
86    networks: 
87      - rabbitmq-net
88    restart: unless-stopped
89
90volumes: 
91  rabbitmq1_data: 
92  rabbitmq2_data: 
93  rabbitmq3_data: 
94  prometheus_data: 
95  grafana_data: 
96
97networks: 
98  rabbitmq-net: 
99    driver: bridge

.env Template

.env

1# RabbitMQ Configuration
2ERLANG_COOKIE=secure_erlang_cookie_value
3RABBITMQ_USER=admin
4RABBITMQ_PASSWORD=secure_rabbitmq_password
5
6# Grafana
7GRAFANA_PASSWORD=secure_grafana_password

Usage Notes

1RabbitMQ Management at http://localhost:15672
2AMQP connection via HAProxy at localhost:5672
3HAProxy stats at http://localhost:8404
4Create rabbitmq.conf with cluster settings

Individual Services(6 services)

Copy individual services to mix and match with your existing compose files.

rabbitmq1

rabbitmq1:
  image: rabbitmq:3-management
  hostname: rabbitmq1
  environment:
    RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
    RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
    RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
  volumes:
    - rabbitmq1_data:/var/lib/rabbitmq
    - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
    - ./definitions.json:/etc/rabbitmq/definitions.json:ro
  networks:
    - rabbitmq-net
  restart: unless-stopped

rabbitmq2

rabbitmq2:
  image: rabbitmq:3-management
  hostname: rabbitmq2
  environment:
    RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
    RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
    RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
  volumes:
    - rabbitmq2_data:/var/lib/rabbitmq
    - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
  depends_on:
    - rabbitmq1
  networks:
    - rabbitmq-net
  restart: unless-stopped

rabbitmq3

rabbitmq3:
  image: rabbitmq:3-management
  hostname: rabbitmq3
  environment:
    RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
    RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
    RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
  volumes:
    - rabbitmq3_data:/var/lib/rabbitmq
    - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
  depends_on:
    - rabbitmq1
  networks:
    - rabbitmq-net
  restart: unless-stopped

haproxy

haproxy:
  image: haproxy:latest
  ports:
    - "5672:5672"
    - "15672:15672"
    - "8404:8404"
  volumes:
    - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
  depends_on:
    - rabbitmq1
    - rabbitmq2
    - rabbitmq3
  networks:
    - rabbitmq-net
  restart: unless-stopped

prometheus

prometheus:
  image: prom/prometheus:latest
  ports:
    - "9090:9090"
  volumes:
    - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
    - prometheus_data:/prometheus
  networks:
    - rabbitmq-net
  restart: unless-stopped

grafana

grafana:
  image: grafana/grafana:latest
  ports:
    - "3000:3000"
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
  volumes:
    - grafana_data:/var/lib/grafana
  depends_on:
    - prometheus
  networks:
    - rabbitmq-net
  restart: unless-stopped

Quick Start

terminal

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  rabbitmq1:
5    image: rabbitmq:3-management
6    hostname: rabbitmq1
7    environment:
8      RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
9      RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
10      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
11    volumes:
12      - rabbitmq1_data:/var/lib/rabbitmq
13      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
14      - ./definitions.json:/etc/rabbitmq/definitions.json:ro
15    networks:
16      - rabbitmq-net
17    restart: unless-stopped
18
19  rabbitmq2:
20    image: rabbitmq:3-management
21    hostname: rabbitmq2
22    environment:
23      RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
24      RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
25      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
26    volumes:
27      - rabbitmq2_data:/var/lib/rabbitmq
28      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
29    depends_on:
30      - rabbitmq1
31    networks:
32      - rabbitmq-net
33    restart: unless-stopped
34
35  rabbitmq3:
36    image: rabbitmq:3-management
37    hostname: rabbitmq3
38    environment:
39      RABBITMQ_ERLANG_COOKIE: ${ERLANG_COOKIE}
40      RABBITMQ_DEFAULT_USER: ${RABBITMQ_USER}
41      RABBITMQ_DEFAULT_PASS: ${RABBITMQ_PASSWORD}
42    volumes:
43      - rabbitmq3_data:/var/lib/rabbitmq
44      - ./rabbitmq.conf:/etc/rabbitmq/rabbitmq.conf:ro
45    depends_on:
46      - rabbitmq1
47    networks:
48      - rabbitmq-net
49    restart: unless-stopped
50
51  haproxy:
52    image: haproxy:latest
53    ports:
54      - "5672:5672"
55      - "15672:15672"
56      - "8404:8404"
57    volumes:
58      - ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
59    depends_on:
60      - rabbitmq1
61      - rabbitmq2
62      - rabbitmq3
63    networks:
64      - rabbitmq-net
65    restart: unless-stopped
66
67  prometheus:
68    image: prom/prometheus:latest
69    ports:
70      - "9090:9090"
71    volumes:
72      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
73      - prometheus_data:/prometheus
74    networks:
75      - rabbitmq-net
76    restart: unless-stopped
77
78  grafana:
79    image: grafana/grafana:latest
80    ports:
81      - "3000:3000"
82    environment:
83      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
84    volumes:
85      - grafana_data:/var/lib/grafana
86    depends_on:
87      - prometheus
88    networks:
89      - rabbitmq-net
90    restart: unless-stopped
91
92volumes:
93  rabbitmq1_data:
94  rabbitmq2_data:
95  rabbitmq3_data:
96  prometheus_data:
97  grafana_data:
98
99networks:
100  rabbitmq-net:
101    driver: bridge
102EOF
103
104# 2. Create the .env file
105cat > .env << 'EOF'
106# RabbitMQ Configuration
107ERLANG_COOKIE=secure_erlang_cookie_value
108RABBITMQ_USER=admin
109RABBITMQ_PASSWORD=secure_rabbitmq_password
110
111# Grafana
112GRAFANA_PASSWORD=secure_grafana_password
113EOF
114
115# 3. Start the services
116docker compose up -d
117
118# 4. View logs
119docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal

1curl -fsSL https://docker.recipes/api/recipes/rabbitmq-ha-cluster/run | bash

Troubleshooting

RabbitMQ nodes failing to join cluster: Verify ERLANG_COOKIE environment variable is identical across all nodes and check hostname resolution
HAProxy showing backend servers as DOWN: Check RabbitMQ management plugin is enabled and nodes are responding on port 15672
Messages not replicating across cluster: Ensure queue mirroring policy is configured in definitions.json or applied via management UI
High memory usage warnings: Adjust RabbitMQ memory thresholds in rabbitmq.conf and monitor queue depths for unbounded growth
Prometheus not scraping RabbitMQ metrics: Verify prometheus_rabbitmq_exporter plugin is enabled and accessible on port 15692
Split-brain cluster scenarios: Implement cluster partition handling strategy and check network connectivity between RabbitMQ nodes

Community Notes

Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

rabbitmqhaproxyprometheusgrafana

Related Recipes

Ad Space

Shortcuts: C CopyF FavoriteD Download