docker.recipes

Netdata Monitoring Cluster

intermediate

Distributed real-time monitoring with Netdata parent-child streaming architecture. Central dashboard aggregates metrics from multiple nodes.

Overview

Netdata is a high-performance, real-time system monitoring and performance analytics platform that provides sub-second granularity metrics collection with zero-configuration auto-detection of system components, applications, and services. Originally developed as an open-source project focused on lightweight resource usage and instant deployment, Netdata has evolved into a comprehensive monitoring solution that can scale from single servers to large distributed infrastructures while maintaining its core philosophy of immediate visibility without complex setup procedures. This deployment creates a distributed monitoring architecture using Netdata's parent-child streaming functionality, where the netdata-parent container acts as a central aggregation hub that receives and stores metrics from remote nodes, while the netdata-child container demonstrates the streaming agent configuration that collects local system metrics and forwards them to the parent node. The parent node provides the primary web dashboard interface and persistent data storage, while child nodes operate in a lightweight streaming mode that minimizes local resource usage and eliminates the need for distributed storage management. This configuration is ideal for system administrators managing multi-server environments, DevOps teams requiring centralized observability across distributed infrastructure, and organizations that need real-time performance monitoring without the complexity and overhead of traditional enterprise monitoring solutions. The parent-child architecture enables horizontal scaling of monitoring coverage while maintaining centralized visualization and alerting capabilities.

Key Features

  • Real-time metrics streaming with sub-second granularity from child nodes to parent aggregator
  • Zero-configuration auto-detection of 800+ system metrics, applications, and services on each monitored node
  • Interactive web dashboard with drill-down capabilities for analyzing metrics across the entire infrastructure
  • Automatic anomaly detection using machine learning algorithms trained on historical metric patterns
  • Custom alerting system with support for multiple notification channels including webhooks and cloud integrations
  • Distributed architecture that reduces bandwidth usage through intelligent metric compression and streaming protocols
  • Built-in health monitoring with configurable thresholds for system components, processes, and custom metrics
  • API key-based authentication for secure metric streaming between geographically distributed nodes

Common Use Cases

  • 1Multi-server application monitoring where developers need unified visibility across web servers, databases, and application nodes
  • 2Container orchestration environments requiring real-time monitoring of Kubernetes clusters or Docker Swarm deployments
  • 3Edge computing deployments where lightweight monitoring agents stream metrics from remote locations to central operations centers
  • 4Development and staging environment monitoring with centralized dashboards for CI/CD pipeline performance analysis
  • 5Hybrid cloud infrastructure monitoring combining on-premises servers with cloud instances under single management interface
  • 6High-frequency trading or real-time application monitoring where sub-second metric resolution is critical for performance optimization
  • 7Managed service provider environments where multiple client infrastructures require centralized monitoring with tenant isolation

Prerequisites

  • Docker host with at least 512MB RAM available for the parent node and 256MB per child node
  • Network connectivity between parent and child nodes on port 19999 for metric streaming
  • Understanding of system monitoring concepts including CPU, memory, disk, and network performance metrics
  • Basic knowledge of UUID generation for creating secure API keys for stream authentication
  • Administrative access to mount host system directories (/proc, /sys) for comprehensive system metric collection
  • Firewall configuration allowing inbound connections to the parent node from all child monitoring nodes

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  netdata-parent: 
3    image: netdata/netdata:latest
4    container_name: netdata-parent
5    restart: unless-stopped
6    hostname: netdata-parent
7    ports: 
8      - "${NETDATA_PORT:-19999}:19999"
9    cap_add: 
10      - SYS_PTRACE
11    security_opt: 
12      - apparmor:unconfined
13    volumes: 
14      - netdata_config:/etc/netdata
15      - netdata_lib:/var/lib/netdata
16      - netdata_cache:/var/cache/netdata
17      - /proc:/host/proc:ro
18      - /sys:/host/sys:ro
19      - /etc/os-release:/host/etc/os-release:ro
20    environment: 
21      - NETDATA_CLAIM_TOKEN=${CLAIM_TOKEN:-}
22      - NETDATA_CLAIM_ROOMS=${CLAIM_ROOMS:-}
23
24  netdata-child: 
25    image: netdata/netdata:latest
26    container_name: netdata-child
27    restart: unless-stopped
28    hostname: netdata-child
29    cap_add: 
30      - SYS_PTRACE
31    security_opt: 
32      - apparmor:unconfined
33    volumes: 
34      - /proc:/host/proc:ro
35      - /sys:/host/sys:ro
36      - /etc/os-release:/host/etc/os-release:ro
37    environment: 
38      - NETDATA_STREAM_DESTINATION=netdata-parent:19999
39      - NETDATA_STREAM_API_KEY=${STREAM_API_KEY}
40
41volumes: 
42  netdata_config: 
43  netdata_lib: 
44  netdata_cache:

.env Template

.env
1# Netdata Cluster Configuration
2NETDATA_PORT=19999
3
4# Generate a UUID for streaming: uuidgen
5STREAM_API_KEY=11111111-2222-3333-4444-555555555555
6
7# Optional: Netdata Cloud claim token (get from app.netdata.cloud)
8CLAIM_TOKEN=
9CLAIM_ROOMS=

Usage Notes

  1. 1Access parent dashboard at http://localhost:19999
  2. 2Child node automatically streams metrics to parent
  3. 3Generate stream API key with: uuidgen
  4. 4Deploy child container on other hosts pointing to parent
  5. 5Optional: claim to Netdata Cloud for alerts and dashboards
  6. 6Add more child nodes by replicating the child service config

Individual Services(2 services)

Copy individual services to mix and match with your existing compose files.

netdata-parent
netdata-parent:
  image: netdata/netdata:latest
  container_name: netdata-parent
  restart: unless-stopped
  hostname: netdata-parent
  ports:
    - ${NETDATA_PORT:-19999}:19999
  cap_add:
    - SYS_PTRACE
  security_opt:
    - apparmor:unconfined
  volumes:
    - netdata_config:/etc/netdata
    - netdata_lib:/var/lib/netdata
    - netdata_cache:/var/cache/netdata
    - /proc:/host/proc:ro
    - /sys:/host/sys:ro
    - /etc/os-release:/host/etc/os-release:ro
  environment:
    - NETDATA_CLAIM_TOKEN=${CLAIM_TOKEN:-}
    - NETDATA_CLAIM_ROOMS=${CLAIM_ROOMS:-}
netdata-child
netdata-child:
  image: netdata/netdata:latest
  container_name: netdata-child
  restart: unless-stopped
  hostname: netdata-child
  cap_add:
    - SYS_PTRACE
  security_opt:
    - apparmor:unconfined
  volumes:
    - /proc:/host/proc:ro
    - /sys:/host/sys:ro
    - /etc/os-release:/host/etc/os-release:ro
  environment:
    - NETDATA_STREAM_DESTINATION=netdata-parent:19999
    - NETDATA_STREAM_API_KEY=${STREAM_API_KEY}

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  netdata-parent:
5    image: netdata/netdata:latest
6    container_name: netdata-parent
7    restart: unless-stopped
8    hostname: netdata-parent
9    ports:
10      - "${NETDATA_PORT:-19999}:19999"
11    cap_add:
12      - SYS_PTRACE
13    security_opt:
14      - apparmor:unconfined
15    volumes:
16      - netdata_config:/etc/netdata
17      - netdata_lib:/var/lib/netdata
18      - netdata_cache:/var/cache/netdata
19      - /proc:/host/proc:ro
20      - /sys:/host/sys:ro
21      - /etc/os-release:/host/etc/os-release:ro
22    environment:
23      - NETDATA_CLAIM_TOKEN=${CLAIM_TOKEN:-}
24      - NETDATA_CLAIM_ROOMS=${CLAIM_ROOMS:-}
25
26  netdata-child:
27    image: netdata/netdata:latest
28    container_name: netdata-child
29    restart: unless-stopped
30    hostname: netdata-child
31    cap_add:
32      - SYS_PTRACE
33    security_opt:
34      - apparmor:unconfined
35    volumes:
36      - /proc:/host/proc:ro
37      - /sys:/host/sys:ro
38      - /etc/os-release:/host/etc/os-release:ro
39    environment:
40      - NETDATA_STREAM_DESTINATION=netdata-parent:19999
41      - NETDATA_STREAM_API_KEY=${STREAM_API_KEY}
42
43volumes:
44  netdata_config:
45  netdata_lib:
46  netdata_cache:
47EOF
48
49# 2. Create the .env file
50cat > .env << 'EOF'
51# Netdata Cluster Configuration
52NETDATA_PORT=19999
53
54# Generate a UUID for streaming: uuidgen
55STREAM_API_KEY=11111111-2222-3333-4444-555555555555
56
57# Optional: Netdata Cloud claim token (get from app.netdata.cloud)
58CLAIM_TOKEN=
59CLAIM_ROOMS=
60EOF
61
62# 3. Start the services
63docker compose up -d
64
65# 4. View logs
66docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/netdata-cluster/run | bash

Troubleshooting

  • Child node shows 'connection refused' errors: Verify netdata-parent container is running and port 19999 is accessible, check NETDATA_STREAM_DESTINATION environment variable format
  • Stream authentication failures between nodes: Regenerate STREAM_API_KEY using uuidgen command and ensure identical keys are configured on both parent and child containers
  • Missing system metrics on monitored nodes: Verify /proc and /sys host directories are properly mounted as read-only volumes in the container configuration
  • High memory usage on netdata-parent container: Reduce metric retention period by configuring history settings or increase available memory allocation for the parent node
  • Dashboard shows 'no data' for child nodes: Check network connectivity between containers and verify child node hostname resolution in the parent container logs
  • Permission denied errors during container startup: Add SYS_PTRACE capability and configure apparmor:unconfined security option for proper system monitoring access

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

netdata-parentnetdata-child

Tags

#netdata#infrastructure#metrics#realtime#distributed

Category

Monitoring & Observability

Related Recipes

Ad Space