docker.recipes

Loki Logging Stack

intermediate

Grafana Loki for log aggregation with Promtail and Grafana.

Overview

Grafana Loki is a horizontally scalable, highly available log aggregation system inspired by Prometheus that revolutionizes log management by indexing only metadata rather than full log content. Unlike traditional log aggregation solutions like Elasticsearch that index entire log messages, Loki uses label-based indexing similar to Prometheus metrics, making it significantly more cost-effective for long-term log storage while maintaining powerful query capabilities through its LogQL query language. This logging stack combines Loki as the central log storage engine, Promtail as the log collection agent, and Grafana for visualization and exploration. Promtail acts as the log shipper that discovers log files on the host system, applies labels based on configured rules, and forwards structured log entries to Loki. Grafana provides the user interface for querying logs using LogQL, creating dashboards that combine both metrics and logs, and setting up alerting rules based on log patterns or volume. This stack is ideal for teams already using Prometheus and Grafana who want to add centralized logging without the complexity and cost of Elasticsearch-based solutions. It's particularly valuable for Kubernetes environments, cost-conscious organizations that need long-term log retention, and teams that prefer the operational simplicity of the Prometheus ecosystem. The combination provides a unified observability platform where metrics, logs, and traces can be correlated in a single interface.

Key Features

  • LogQL query language for powerful log filtering and aggregation similar to PromQL
  • Label-based indexing that dramatically reduces storage costs compared to full-text indexing
  • Native Grafana integration allowing log and metric correlation in unified dashboards
  • Promtail automatic service discovery for Docker containers and Kubernetes pods
  • Multi-tenant architecture supporting multiple organizations with data isolation
  • Horizontal scalability with support for object storage backends like S3, GCS, and Azure
  • Real-time log streaming and alerting based on log patterns and volume thresholds
  • Compression and chunk-based storage optimization for efficient long-term retention

Common Use Cases

  • 1Kubernetes cluster log aggregation with automatic pod discovery and labeling
  • 2Cost-effective alternative to Elasticsearch for organizations with high log volumes
  • 3Unified observability platform combining Prometheus metrics with centralized logging
  • 4Application debugging and troubleshooting with structured log queries
  • 5Security monitoring and audit log analysis using label-based filtering
  • 6DevOps teams needing correlation between application performance metrics and logs
  • 7Multi-environment log management for development, staging, and production systems

Prerequisites

  • Minimum 1.5GB RAM available (256MB for Loki, 512MB for Grafana, 256MB for Promtail)
  • Docker and Docker Compose v2.0 or higher installed
  • Ports 3000 (Grafana) and 3100 (Loki) available on the host system
  • Basic understanding of log aggregation concepts and Grafana dashboard creation
  • Read access to system log directories (promtail-config.yml must be created)
  • Sufficient disk space for log storage (Loki data volume grows with log retention)

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  loki: 
3    image: grafana/loki:latest
4    container_name: loki
5    restart: unless-stopped
6    ports: 
7      - "${LOKI_PORT:-3100}:3100"
8    command: -config.file=/etc/loki/local-config.yaml
9    volumes: 
10      - loki_data:/loki
11    networks: 
12      - loki-network
13
14  promtail: 
15    image: grafana/promtail:latest
16    container_name: promtail
17    restart: unless-stopped
18    volumes: 
19      - /var/log:/var/log:ro
20      - ./promtail-config.yml:/etc/promtail/config.yml:ro
21    command: -config.file=/etc/promtail/config.yml
22    depends_on: 
23      - loki
24    networks: 
25      - loki-network
26
27  grafana: 
28    image: grafana/grafana:latest
29    container_name: grafana-loki
30    restart: unless-stopped
31    ports: 
32      - "${GRAFANA_PORT:-3000}:3000"
33    environment: 
34      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD:-admin}
35    volumes: 
36      - grafana_data:/var/lib/grafana
37    depends_on: 
38      - loki
39    networks: 
40      - loki-network
41
42volumes: 
43  loki_data: 
44  grafana_data: 
45
46networks: 
47  loki-network: 
48    driver: bridge

.env Template

.env
1# Loki Stack
2LOKI_PORT=3100
3GRAFANA_PORT=3000
4GRAFANA_PASSWORD=admin

Usage Notes

  1. 1Grafana at http://localhost:3000
  2. 2Loki at http://localhost:3100
  3. 3Add Loki datasource in Grafana
  4. 4Create promtail-config.yml

Individual Services(3 services)

Copy individual services to mix and match with your existing compose files.

loki
loki:
  image: grafana/loki:latest
  container_name: loki
  restart: unless-stopped
  ports:
    - ${LOKI_PORT:-3100}:3100
  command: "-config.file=/etc/loki/local-config.yaml"
  volumes:
    - loki_data:/loki
  networks:
    - loki-network
promtail
promtail:
  image: grafana/promtail:latest
  container_name: promtail
  restart: unless-stopped
  volumes:
    - /var/log:/var/log:ro
    - ./promtail-config.yml:/etc/promtail/config.yml:ro
  command: "-config.file=/etc/promtail/config.yml"
  depends_on:
    - loki
  networks:
    - loki-network
grafana
grafana:
  image: grafana/grafana:latest
  container_name: grafana-loki
  restart: unless-stopped
  ports:
    - ${GRAFANA_PORT:-3000}:3000
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD:-admin}
  volumes:
    - grafana_data:/var/lib/grafana
  depends_on:
    - loki
  networks:
    - loki-network

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  loki:
5    image: grafana/loki:latest
6    container_name: loki
7    restart: unless-stopped
8    ports:
9      - "${LOKI_PORT:-3100}:3100"
10    command: -config.file=/etc/loki/local-config.yaml
11    volumes:
12      - loki_data:/loki
13    networks:
14      - loki-network
15
16  promtail:
17    image: grafana/promtail:latest
18    container_name: promtail
19    restart: unless-stopped
20    volumes:
21      - /var/log:/var/log:ro
22      - ./promtail-config.yml:/etc/promtail/config.yml:ro
23    command: -config.file=/etc/promtail/config.yml
24    depends_on:
25      - loki
26    networks:
27      - loki-network
28
29  grafana:
30    image: grafana/grafana:latest
31    container_name: grafana-loki
32    restart: unless-stopped
33    ports:
34      - "${GRAFANA_PORT:-3000}:3000"
35    environment:
36      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD:-admin}
37    volumes:
38      - grafana_data:/var/lib/grafana
39    depends_on:
40      - loki
41    networks:
42      - loki-network
43
44volumes:
45  loki_data:
46  grafana_data:
47
48networks:
49  loki-network:
50    driver: bridge
51EOF
52
53# 2. Create the .env file
54cat > .env << 'EOF'
55# Loki Stack
56LOKI_PORT=3100
57GRAFANA_PORT=3000
58GRAFANA_PASSWORD=admin
59EOF
60
61# 3. Start the services
62docker compose up -d
63
64# 4. View logs
65docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/loki-logging-stack/run | bash

Troubleshooting

  • Promtail shows 'permission denied' errors: Ensure Docker has read access to /var/log or adjust volume mounts to accessible directories
  • Loki container exits with 'failed to parse config': Verify the built-in local-config.yaml is not overridden by custom configurations
  • Grafana shows 'Bad Gateway' when querying Loki: Check that Loki datasource URL is set to http://loki:3100 using the Docker network name
  • No logs appearing in Grafana: Create and mount promtail-config.yml with proper scrape_configs pointing to log file paths
  • Loki runs out of disk space: Configure log retention policies in Loki config or set up object storage backend for longer retention
  • Promtail not discovering containers: Add Docker socket mount and configure docker_sd_config in promtail configuration

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

lokipromtailgrafana

Tags

#loki#logging#grafana#promtail#prometheus

Category

Monitoring & Observability

Related Recipes

Ad Space