docker.recipes

NATS + JetStream + Monitoring

intermediate

Cloud native messaging system with persistence.

Overview

NATS is a high-performance, cloud-native messaging system originally developed by Derek Collison in 2010, designed to be simple, secure, and performant for modern distributed applications. Unlike traditional message brokers that focus on complex routing and delivery guarantees, NATS prioritizes speed and simplicity, making it ideal for microservices architectures, IoT systems, and real-time applications where low latency is critical. This stack combines NATS with JetStream persistence capabilities, NATS Surveyor for metrics collection, Prometheus for time-series monitoring, and Grafana for visualization. JetStream transforms NATS from a pure fire-and-forget messaging system into one with durable streams, consumer groups, and exactly-once delivery semantics. NATS Surveyor acts as a specialized metrics exporter that understands NATS internals, feeding connection counts, message rates, and JetStream consumer lag into Prometheus for alerting and historical analysis. Development teams building event-driven architectures, platform engineers managing microservices communication, and organizations migrating from heavier message brokers like RabbitMQ will find this stack particularly valuable. The monitoring layer provides crucial visibility into message throughput, consumer health, and system performance that's essential for production NATS deployments, especially when using JetStream's persistence features for critical business events.

Key Features

  • JetStream persistence with configurable retention policies and replay capabilities
  • Pub/Sub messaging with subject-based routing and wildcard subscriptions
  • Request/Reply pattern for synchronous communication between services
  • Queue groups for load balancing messages across consumer instances
  • NATS Surveyor metrics collection with account-level observability
  • Built-in key-value store using JetStream for configuration and state management
  • WebSocket support for browser-based clients and real-time web applications
  • Multi-tenancy through NATS accounts with isolated message spaces

Common Use Cases

  • 1Microservices event bus for order processing and inventory management systems
  • 2IoT telemetry collection with JetStream persistence for offline device recovery
  • 3Real-time chat applications using NATS WebSocket gateway and subject filtering
  • 4Financial trading systems requiring ultra-low latency market data distribution
  • 5DevOps automation pipelines with reliable job queuing and status updates
  • 6Gaming backends for matchmaking, leaderboards, and real-time player interactions
  • 7Monitoring agent coordination across distributed infrastructure deployments

Prerequisites

  • Minimum 512MB RAM available (256MB for NATS, 256MB for monitoring stack)
  • Ports 3000, 4222, 7777, 8222, and 9090 available on the host system
  • Basic understanding of pub/sub messaging patterns and subject hierarchies
  • Familiarity with Prometheus metrics and PromQL query language
  • Docker Compose 3.8+ with volume and network support
  • Understanding of JetStream concepts like streams, consumers, and acknowledgments

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  nats: 
3    image: nats:latest
4    command: 
5      - "--jetstream"
6      - "--store_dir=/data"
7      - "--http_port=8222"
8    volumes: 
9      - nats-data:/data
10    ports: 
11      - "4222:4222"
12      - "8222:8222"
13    networks: 
14      - nats-network
15    restart: unless-stopped
16
17  nats-surveyor: 
18    image: natsio/nats-surveyor:latest
19    command: 
20      - "-s=nats://nats:4222"
21      - "--accounts"
22      - "--observe=*"
23    ports: 
24      - "7777:7777"
25    depends_on: 
26      - nats
27    networks: 
28      - nats-network
29    restart: unless-stopped
30
31  prometheus: 
32    image: prom/prometheus:latest
33    command: 
34      - --config.file=/etc/prometheus/prometheus.yml
35      - --storage.tsdb.path=/prometheus
36    volumes: 
37      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
38      - prometheus-data:/prometheus
39    ports: 
40      - "9090:9090"
41    depends_on: 
42      - nats-surveyor
43    networks: 
44      - nats-network
45    restart: unless-stopped
46
47  grafana: 
48    image: grafana/grafana:latest
49    environment: 
50      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
51    volumes: 
52      - grafana-data:/var/lib/grafana
53    ports: 
54      - "3000:3000"
55    depends_on: 
56      - prometheus
57    networks: 
58      - nats-network
59    restart: unless-stopped
60
61volumes: 
62  nats-data: 
63  prometheus-data: 
64  grafana-data: 
65
66networks: 
67  nats-network: 
68    driver: bridge

.env Template

.env
1# NATS
2GRAFANA_PASSWORD=secure_grafana_password
3
4# NATS at nats://localhost:4222
5# Monitoring at http://localhost:8222

Usage Notes

  1. 1NATS at nats://localhost:4222
  2. 2Monitoring at http://localhost:8222
  3. 3Grafana at http://localhost:3000
  4. 4JetStream for persistence
  5. 5Built-in key-value store

Individual Services(4 services)

Copy individual services to mix and match with your existing compose files.

nats
nats:
  image: nats:latest
  command:
    - "--jetstream"
    - "--store_dir=/data"
    - "--http_port=8222"
  volumes:
    - nats-data:/data
  ports:
    - "4222:4222"
    - "8222:8222"
  networks:
    - nats-network
  restart: unless-stopped
nats-surveyor
nats-surveyor:
  image: natsio/nats-surveyor:latest
  command:
    - "-s=nats://nats:4222"
    - "--accounts"
    - "--observe=*"
  ports:
    - "7777:7777"
  depends_on:
    - nats
  networks:
    - nats-network
  restart: unless-stopped
prometheus
prometheus:
  image: prom/prometheus:latest
  command:
    - "--config.file=/etc/prometheus/prometheus.yml"
    - "--storage.tsdb.path=/prometheus"
  volumes:
    - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
    - prometheus-data:/prometheus
  ports:
    - "9090:9090"
  depends_on:
    - nats-surveyor
  networks:
    - nats-network
  restart: unless-stopped
grafana
grafana:
  image: grafana/grafana:latest
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
  volumes:
    - grafana-data:/var/lib/grafana
  ports:
    - "3000:3000"
  depends_on:
    - prometheus
  networks:
    - nats-network
  restart: unless-stopped

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  nats:
5    image: nats:latest
6    command:
7      - "--jetstream"
8      - "--store_dir=/data"
9      - "--http_port=8222"
10    volumes:
11      - nats-data:/data
12    ports:
13      - "4222:4222"
14      - "8222:8222"
15    networks:
16      - nats-network
17    restart: unless-stopped
18
19  nats-surveyor:
20    image: natsio/nats-surveyor:latest
21    command:
22      - "-s=nats://nats:4222"
23      - "--accounts"
24      - "--observe=*"
25    ports:
26      - "7777:7777"
27    depends_on:
28      - nats
29    networks:
30      - nats-network
31    restart: unless-stopped
32
33  prometheus:
34    image: prom/prometheus:latest
35    command:
36      - --config.file=/etc/prometheus/prometheus.yml
37      - --storage.tsdb.path=/prometheus
38    volumes:
39      - ./prometheus.yml:/etc/prometheus/prometheus.yml:ro
40      - prometheus-data:/prometheus
41    ports:
42      - "9090:9090"
43    depends_on:
44      - nats-surveyor
45    networks:
46      - nats-network
47    restart: unless-stopped
48
49  grafana:
50    image: grafana/grafana:latest
51    environment:
52      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
53    volumes:
54      - grafana-data:/var/lib/grafana
55    ports:
56      - "3000:3000"
57    depends_on:
58      - prometheus
59    networks:
60      - nats-network
61    restart: unless-stopped
62
63volumes:
64  nats-data:
65  prometheus-data:
66  grafana-data:
67
68networks:
69  nats-network:
70    driver: bridge
71EOF
72
73# 2. Create the .env file
74cat > .env << 'EOF'
75# NATS
76GRAFANA_PASSWORD=secure_grafana_password
77
78# NATS at nats://localhost:4222
79# Monitoring at http://localhost:8222
80EOF
81
82# 3. Start the services
83docker compose up -d
84
85# 4. View logs
86docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/nats-complete/run | bash

Troubleshooting

  • Error 'nats: no servers available for connection': Check that NATS container is running and port 4222 is accessible
  • JetStream 'insufficient resources' error: Increase memory limits or reduce stream retention settings in NATS configuration
  • NATS Surveyor showing 0 connections: Verify the connection string 'nats://nats:4222' and ensure containers are on the same network
  • Grafana showing 'no data points' for NATS metrics: Check Prometheus targets page and verify nats-surveyor:7777 is being scraped successfully
  • High memory usage in NATS container: Review JetStream stream configurations and set appropriate max_age or max_msgs limits
  • Consumer lag alerts firing: Check if consumer applications are processing messages and consider scaling consumer instances

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

natsnats-surveyorprometheusgrafana

Tags

#nats#messaging#jetstream#cloud-native#pub-sub

Category

Message Queues & Brokers

Related Recipes

Ad Space