HashiCorp Vault Secrets Management

advanced

Secrets management with Vault HA cluster, Consul backend, and UI.

Overview

HashiCorp Vault is an industry-leading secrets management platform designed to secure, store, and tightly control access to tokens, passwords, certificates, API keys, and other sensitive data. Originally developed by HashiCorp in 2015, Vault addresses the critical security challenge of secrets sprawl in modern infrastructure, where applications and systems require access to numerous credentials and encryption keys. Unlike traditional password managers or basic key stores, Vault provides dynamic secrets generation, automatic credential rotation, and fine-grained policy-based access controls that integrate with existing identity systems. This production-grade stack combines Vault with HashiCorp Consul as a highly available storage backend, creating a resilient cluster that can survive individual node failures without losing access to critical secrets. Consul provides distributed consensus through the Raft algorithm and service discovery capabilities, while Grafana and Prometheus deliver comprehensive monitoring of vault operations, unsealing events, authentication attempts, and performance metrics. The three-node Vault cluster ensures no single point of failure, with automatic leadership election and data replication across all instances. This configuration targets organizations implementing zero-trust security models, DevOps teams managing multi-cloud deployments, and enterprises requiring compliance with regulations like SOX, PCI-DSS, or GDPR. The combination of Vault's encryption-as-a-service capabilities with Consul's distributed architecture makes this stack particularly valuable for companies running microservices architectures where traditional secrets management approaches become unwieldy and insecure.

Key Features

High availability three-node Vault cluster with automatic failover and leader election
Consul-backed distributed storage with Raft consensus for data consistency
Dynamic secrets generation for databases, cloud providers, and SSH access
Transit encryption engine for encryption-as-a-service operations
PKI engine for automated certificate lifecycle management
Policy-based access control with fine-grained permissions and audit logging
Prometheus metrics collection for vault operations and cluster health
Grafana dashboards for monitoring unsealing events, authentication patterns, and secret access

Common Use Cases

1Enterprise secrets management replacing hardcoded credentials in applications
2Database credential rotation for PostgreSQL, MySQL, and MongoDB instances
3PKI infrastructure for internal certificate authority and automated cert provisioning
4Multi-cloud API key management across AWS, Azure, and Google Cloud Platform
5Kubernetes secrets injection using vault-agent or CSI drivers
6Compliance environments requiring audit trails and encryption key management
7DevOps pipelines needing secure credential distribution to CI/CD systems

Prerequisites

Docker Engine 20.10+ with at least 4GB available memory for the complete stack
Basic understanding of HashiCorp Vault concepts including unsealing and root tokens
Network access to ports 8200 (Vault UI), 8500 (Consul UI), 9090 (Prometheus), and 3000 (Grafana)
Familiarity with Vault CLI commands for initialization and configuration
Understanding of high availability concepts and distributed systems basics

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

1services: 
2  consul: 
3    image: hashicorp/consul:latest
4    command: agent -server -bootstrap-expect=1 -ui -client=0.0.0.0
5    ports: 
6      - "8500:8500"
7    volumes: 
8      - consul_data:/consul/data
9    networks: 
10      - vault_net
11
12  vault-1: 
13    image: hashicorp/vault:latest
14    cap_add: 
15      - IPC_LOCK
16    environment: 
17      - VAULT_ADDR=http://0.0.0.0:8200
18      - VAULT_API_ADDR=http://vault-1:8200
19      - VAULT_CLUSTER_ADDR=http://vault-1:8201
20    volumes: 
21      - ./vault/config.hcl:/vault/config/config.hcl
22      - vault_1:/vault/data
23    command: server
24    ports: 
25      - "8200:8200"
26    depends_on: 
27      - consul
28    networks: 
29      - vault_net
30
31  vault-2: 
32    image: hashicorp/vault:latest
33    cap_add: 
34      - IPC_LOCK
35    environment: 
36      - VAULT_ADDR=http://0.0.0.0:8200
37      - VAULT_API_ADDR=http://vault-2:8200
38      - VAULT_CLUSTER_ADDR=http://vault-2:8201
39    volumes: 
40      - ./vault/config.hcl:/vault/config/config.hcl
41      - vault_2:/vault/data
42    command: server
43    depends_on: 
44      - consul
45    networks: 
46      - vault_net
47
48  vault-3: 
49    image: hashicorp/vault:latest
50    cap_add: 
51      - IPC_LOCK
52    environment: 
53      - VAULT_ADDR=http://0.0.0.0:8200
54      - VAULT_API_ADDR=http://vault-3:8200
55      - VAULT_CLUSTER_ADDR=http://vault-3:8201
56    volumes: 
57      - ./vault/config.hcl:/vault/config/config.hcl
58      - vault_3:/vault/data
59    command: server
60    depends_on: 
61      - consul
62    networks: 
63      - vault_net
64
65  prometheus: 
66    image: prom/prometheus:latest
67    ports: 
68      - "9090:9090"
69    volumes: 
70      - prometheus_data:/prometheus
71    networks: 
72      - vault_net
73
74  grafana: 
75    image: grafana/grafana:latest
76    ports: 
77      - "3000:3000"
78    environment: 
79      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
80    volumes: 
81      - grafana_data:/var/lib/grafana
82    networks: 
83      - vault_net
84
85volumes: 
86  consul_data: 
87  vault_1: 
88  vault_2: 
89  vault_3: 
90  prometheus_data: 
91  grafana_data: 
92
93networks: 
94  vault_net:

.env Template

.env

1# Vault Cluster
2GRAFANA_PASSWORD=secure_grafana_password
3
4# Vault at http://localhost:8200
5# Initialize with: vault operator init

Usage Notes

1Vault UI at http://localhost:8200
2Initialize with vault operator init
3Consul backend for HA storage
4PKI, transit, and KV engines
5Auto-unseal in production

Individual Services(6 services)

Copy individual services to mix and match with your existing compose files.

consul

consul:
  image: hashicorp/consul:latest
  command: agent -server -bootstrap-expect=1 -ui -client=0.0.0.0
  ports:
    - "8500:8500"
  volumes:
    - consul_data:/consul/data
  networks:
    - vault_net

vault-1

vault-1:
  image: hashicorp/vault:latest
  cap_add:
    - IPC_LOCK
  environment:
    - VAULT_ADDR=http://0.0.0.0:8200
    - VAULT_API_ADDR=http://vault-1:8200
    - VAULT_CLUSTER_ADDR=http://vault-1:8201
  volumes:
    - ./vault/config.hcl:/vault/config/config.hcl
    - vault_1:/vault/data
  command: server
  ports:
    - "8200:8200"
  depends_on:
    - consul
  networks:
    - vault_net

vault-2

vault-2:
  image: hashicorp/vault:latest
  cap_add:
    - IPC_LOCK
  environment:
    - VAULT_ADDR=http://0.0.0.0:8200
    - VAULT_API_ADDR=http://vault-2:8200
    - VAULT_CLUSTER_ADDR=http://vault-2:8201
  volumes:
    - ./vault/config.hcl:/vault/config/config.hcl
    - vault_2:/vault/data
  command: server
  depends_on:
    - consul
  networks:
    - vault_net

vault-3

vault-3:
  image: hashicorp/vault:latest
  cap_add:
    - IPC_LOCK
  environment:
    - VAULT_ADDR=http://0.0.0.0:8200
    - VAULT_API_ADDR=http://vault-3:8200
    - VAULT_CLUSTER_ADDR=http://vault-3:8201
  volumes:
    - ./vault/config.hcl:/vault/config/config.hcl
    - vault_3:/vault/data
  command: server
  depends_on:
    - consul
  networks:
    - vault_net

prometheus

prometheus:
  image: prom/prometheus:latest
  ports:
    - "9090:9090"
  volumes:
    - prometheus_data:/prometheus
  networks:
    - vault_net

grafana

grafana:
  image: grafana/grafana:latest
  ports:
    - "3000:3000"
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
  volumes:
    - grafana_data:/var/lib/grafana
  networks:
    - vault_net

Quick Start

terminal

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  consul:
5    image: hashicorp/consul:latest
6    command: agent -server -bootstrap-expect=1 -ui -client=0.0.0.0
7    ports:
8      - "8500:8500"
9    volumes:
10      - consul_data:/consul/data
11    networks:
12      - vault_net
13
14  vault-1:
15    image: hashicorp/vault:latest
16    cap_add:
17      - IPC_LOCK
18    environment:
19      - VAULT_ADDR=http://0.0.0.0:8200
20      - VAULT_API_ADDR=http://vault-1:8200
21      - VAULT_CLUSTER_ADDR=http://vault-1:8201
22    volumes:
23      - ./vault/config.hcl:/vault/config/config.hcl
24      - vault_1:/vault/data
25    command: server
26    ports:
27      - "8200:8200"
28    depends_on:
29      - consul
30    networks:
31      - vault_net
32
33  vault-2:
34    image: hashicorp/vault:latest
35    cap_add:
36      - IPC_LOCK
37    environment:
38      - VAULT_ADDR=http://0.0.0.0:8200
39      - VAULT_API_ADDR=http://vault-2:8200
40      - VAULT_CLUSTER_ADDR=http://vault-2:8201
41    volumes:
42      - ./vault/config.hcl:/vault/config/config.hcl
43      - vault_2:/vault/data
44    command: server
45    depends_on:
46      - consul
47    networks:
48      - vault_net
49
50  vault-3:
51    image: hashicorp/vault:latest
52    cap_add:
53      - IPC_LOCK
54    environment:
55      - VAULT_ADDR=http://0.0.0.0:8200
56      - VAULT_API_ADDR=http://vault-3:8200
57      - VAULT_CLUSTER_ADDR=http://vault-3:8201
58    volumes:
59      - ./vault/config.hcl:/vault/config/config.hcl
60      - vault_3:/vault/data
61    command: server
62    depends_on:
63      - consul
64    networks:
65      - vault_net
66
67  prometheus:
68    image: prom/prometheus:latest
69    ports:
70      - "9090:9090"
71    volumes:
72      - prometheus_data:/prometheus
73    networks:
74      - vault_net
75
76  grafana:
77    image: grafana/grafana:latest
78    ports:
79      - "3000:3000"
80    environment:
81      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
82    volumes:
83      - grafana_data:/var/lib/grafana
84    networks:
85      - vault_net
86
87volumes:
88  consul_data:
89  vault_1:
90  vault_2:
91  vault_3:
92  prometheus_data:
93  grafana_data:
94
95networks:
96  vault_net:
97EOF
98
99# 2. Create the .env file
100cat > .env << 'EOF'
101# Vault Cluster
102GRAFANA_PASSWORD=secure_grafana_password
103
104# Vault at http://localhost:8200
105# Initialize with: vault operator init
106EOF
107
108# 3. Start the services
109docker compose up -d
110
111# 4. View logs
112docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal

1curl -fsSL https://docker.recipes/api/recipes/vault-cluster/run | bash

Troubleshooting

Vault shows 'sealed' status after restart: Run 'vault operator unseal' with threshold number of unseal keys on each node
Consul connection refused errors: Verify Consul container started successfully and is accessible on port 8500
Permission denied when accessing vault data: Check that containers have IPC_LOCK capability and proper volume mounts
Vault cluster split-brain condition: Restart all Vault nodes simultaneously to force re-election
Prometheus not scraping vault metrics: Configure Vault telemetry in config.hcl and verify prometheus_retention_time setting
Grafana showing no vault data: Import HashiCorp Vault dashboard templates and configure Prometheus data source correctly

Community Notes

Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

vaultconsulnginxprometheus

Related Recipes

Ad Space

Shortcuts: C CopyF FavoriteD Download