docker.recipes

HashiCorp Consul Cluster

advanced

Service mesh and service discovery platform for distributed systems.

Overview

HashiCorp Consul is a distributed service mesh and service discovery platform designed to connect and secure services across any runtime platform and public or private cloud. Originally developed by HashiCorp in 2014, Consul provides a full-featured control plane with service discovery, configuration, and segmentation functionality, enabling organizations to build secure, resilient distributed systems with automatic network configurations and service-to-service communication. This three-server Consul cluster configuration establishes a highly available service registry with automated leader election and consensus through the Raft protocol. The consul-server nodes form a quorum-based cluster that maintains service catalogs, health checks, and key-value data, while the consul-client acts as a lightweight agent for service registration and discovery. Together, they create a distributed system that can handle node failures while maintaining service visibility and enabling secure inter-service communication through built-in service mesh capabilities. This stack is ideal for platform engineering teams managing microservices architectures, DevOps engineers implementing service discovery in multi-cloud environments, and organizations requiring centralized configuration management with high availability. The combination of multiple server nodes with client agents provides the redundancy needed for production workloads while offering the flexibility to scale service registration across distributed applications.

Key Features

  • Raft consensus algorithm for distributed leader election and data replication
  • Built-in service discovery with DNS and HTTP interfaces on ports 8600 and 8500
  • Integrated health checking with configurable check intervals and failure thresholds
  • Key-value store for dynamic configuration and service metadata
  • Service mesh capabilities with Connect for encrypted service-to-service communication
  • Multi-datacenter federation support for cross-region service discovery
  • Access Control Lists (ACL) system for fine-grained service and data permissions
  • Web UI dashboard for cluster visualization and service monitoring

Common Use Cases

  • 1Microservices service discovery in Kubernetes or Docker Swarm environments
  • 2Multi-cloud service registry for applications spanning AWS, Azure, and GCP
  • 3Configuration management for distributed applications requiring dynamic updates
  • 4Service mesh implementation for zero-trust networking between microservices
  • 5Load balancer backend discovery for HAProxy, NGINX, or cloud load balancers
  • 6Cross-datacenter service communication for disaster recovery scenarios
  • 7API gateway service registration for Kong, Ambassador, or Istio gateways

Prerequisites

  • Minimum 1.5GB RAM total (512MB per consul-server node recommended)
  • Docker Engine 20.10+ with Docker Compose v2 support
  • Available ports 8500 (HTTP API/UI) and 8600 (DNS) on the host system
  • Understanding of distributed systems concepts like consensus and leader election
  • Knowledge of service discovery patterns and health check configurations
  • Network connectivity between all cluster nodes for gossip protocol communication

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  consul-server1: 
3    image: hashicorp/consul:latest
4    container_name: consul-server1
5    command: agent -server -bootstrap-expect=3 -ui -client=0.0.0.0 -retry-join=consul-server2 -retry-join=consul-server3
6    environment: 
7      - CONSUL_BIND_INTERFACE=eth0
8    volumes: 
9      - consul-server1:/consul/data
10    ports: 
11      - "8500:8500"
12      - "8600:8600/udp"
13    networks: 
14      - consul-network
15    restart: unless-stopped
16
17  consul-server2: 
18    image: hashicorp/consul:latest
19    container_name: consul-server2
20    command: agent -server -bootstrap-expect=3 -retry-join=consul-server1 -retry-join=consul-server3
21    environment: 
22      - CONSUL_BIND_INTERFACE=eth0
23    volumes: 
24      - consul-server2:/consul/data
25    networks: 
26      - consul-network
27    restart: unless-stopped
28
29  consul-server3: 
30    image: hashicorp/consul:latest
31    container_name: consul-server3
32    command: agent -server -bootstrap-expect=3 -retry-join=consul-server1 -retry-join=consul-server2
33    environment: 
34      - CONSUL_BIND_INTERFACE=eth0
35    volumes: 
36      - consul-server3:/consul/data
37    networks: 
38      - consul-network
39    restart: unless-stopped
40
41  consul-client: 
42    image: hashicorp/consul:latest
43    container_name: consul-client
44    command: agent -retry-join=consul-server1
45    environment: 
46      - CONSUL_BIND_INTERFACE=eth0
47    volumes: 
48      - consul-client:/consul/data
49    networks: 
50      - consul-network
51    restart: unless-stopped
52
53volumes: 
54  consul-server1: 
55  consul-server2: 
56  consul-server3: 
57  consul-client: 
58
59networks: 
60  consul-network: 
61    driver: bridge

.env Template

.env
1# HashiCorp Consul Cluster
2# 3-node server cluster with 1 client

Usage Notes

  1. 1Consul UI at http://localhost:8500
  2. 23-node HA cluster
  3. 3Register services via API or config
  4. 4DNS interface at :8600
  5. 5Health checking built-in

Individual Services(4 services)

Copy individual services to mix and match with your existing compose files.

consul-server1
consul-server1:
  image: hashicorp/consul:latest
  container_name: consul-server1
  command: agent -server -bootstrap-expect=3 -ui -client=0.0.0.0 -retry-join=consul-server2 -retry-join=consul-server3
  environment:
    - CONSUL_BIND_INTERFACE=eth0
  volumes:
    - consul-server1:/consul/data
  ports:
    - "8500:8500"
    - 8600:8600/udp
  networks:
    - consul-network
  restart: unless-stopped
consul-server2
consul-server2:
  image: hashicorp/consul:latest
  container_name: consul-server2
  command: agent -server -bootstrap-expect=3 -retry-join=consul-server1 -retry-join=consul-server3
  environment:
    - CONSUL_BIND_INTERFACE=eth0
  volumes:
    - consul-server2:/consul/data
  networks:
    - consul-network
  restart: unless-stopped
consul-server3
consul-server3:
  image: hashicorp/consul:latest
  container_name: consul-server3
  command: agent -server -bootstrap-expect=3 -retry-join=consul-server1 -retry-join=consul-server2
  environment:
    - CONSUL_BIND_INTERFACE=eth0
  volumes:
    - consul-server3:/consul/data
  networks:
    - consul-network
  restart: unless-stopped
consul-client
consul-client:
  image: hashicorp/consul:latest
  container_name: consul-client
  command: agent -retry-join=consul-server1
  environment:
    - CONSUL_BIND_INTERFACE=eth0
  volumes:
    - consul-client:/consul/data
  networks:
    - consul-network
  restart: unless-stopped

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  consul-server1:
5    image: hashicorp/consul:latest
6    container_name: consul-server1
7    command: agent -server -bootstrap-expect=3 -ui -client=0.0.0.0 -retry-join=consul-server2 -retry-join=consul-server3
8    environment:
9      - CONSUL_BIND_INTERFACE=eth0
10    volumes:
11      - consul-server1:/consul/data
12    ports:
13      - "8500:8500"
14      - "8600:8600/udp"
15    networks:
16      - consul-network
17    restart: unless-stopped
18
19  consul-server2:
20    image: hashicorp/consul:latest
21    container_name: consul-server2
22    command: agent -server -bootstrap-expect=3 -retry-join=consul-server1 -retry-join=consul-server3
23    environment:
24      - CONSUL_BIND_INTERFACE=eth0
25    volumes:
26      - consul-server2:/consul/data
27    networks:
28      - consul-network
29    restart: unless-stopped
30
31  consul-server3:
32    image: hashicorp/consul:latest
33    container_name: consul-server3
34    command: agent -server -bootstrap-expect=3 -retry-join=consul-server1 -retry-join=consul-server2
35    environment:
36      - CONSUL_BIND_INTERFACE=eth0
37    volumes:
38      - consul-server3:/consul/data
39    networks:
40      - consul-network
41    restart: unless-stopped
42
43  consul-client:
44    image: hashicorp/consul:latest
45    container_name: consul-client
46    command: agent -retry-join=consul-server1
47    environment:
48      - CONSUL_BIND_INTERFACE=eth0
49    volumes:
50      - consul-client:/consul/data
51    networks:
52      - consul-network
53    restart: unless-stopped
54
55volumes:
56  consul-server1:
57  consul-server2:
58  consul-server3:
59  consul-client:
60
61networks:
62  consul-network:
63    driver: bridge
64EOF
65
66# 2. Create the .env file
67cat > .env << 'EOF'
68# HashiCorp Consul Cluster
69# 3-node server cluster with 1 client
70EOF
71
72# 3. Start the services
73docker compose up -d
74
75# 4. View logs
76docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/hashicorp-consul/run | bash

Troubleshooting

  • Error 'No cluster leader': Ensure all three server nodes are running and can communicate on port 8300
  • UI shows 'failed' health checks: Verify service definitions include proper health check endpoints and intervals
  • Client cannot join cluster: Check that consul-client can resolve consul-server1 hostname via Docker networking
  • Split-brain scenario after network partition: Restart minority nodes to rejoin the majority partition
  • High memory usage on server nodes: Reduce log level or increase cleanup intervals in consul configuration
  • Service registration fails with ACL errors: Ensure proper ACL tokens are configured if ACL system is enabled

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

consul-serverconsul-clientconsul-envoy

Tags

#service-mesh#consul#hashicorp#service-discovery#networking

Category

DevOps & CI/CD

Related Recipes

Ad Space