# Graylog

> Centralized log management with search and analysis capabilities.

- Category: Monitoring & Observability
- Difficulty: intermediate
- Components: graylog, mongodb, opensearch
- Tags: graylog, logs, centralized, search, analysis
- Source: https://docker.recipes/monitoring/graylog

## Overview

Graylog is an enterprise-grade centralized log management platform that aggregates, indexes, and analyzes log data from across your infrastructure. Originally created in 2010 by Lennart Koopmann, Graylog has evolved into a comprehensive solution for collecting structured and unstructured data from servers, applications, network devices, and cloud services, providing real-time search capabilities and powerful analytics to help organizations maintain visibility into their systems. This stack combines Graylog with MongoDB for metadata storage and OpenSearch for full-text indexing and search functionality. MongoDB stores Graylog's configuration, user accounts, dashboards, and metadata, while OpenSearch handles the heavy lifting of indexing log messages and providing lightning-fast search capabilities across millions of log entries. OpenSearch, Amazon's open-source fork of Elasticsearch, delivers the scalable search engine that powers Graylog's query interface and enables complex log analysis. This three-tier architecture is ideal for organizations that need robust log management without the complexity of managing separate logging solutions. DevOps teams, security analysts, and system administrators benefit from having centralized visibility into application behavior, security events, and system performance metrics, while the web-based interface makes log analysis accessible to both technical and non-technical team members.

## Features

- GELF (Graylog Extended Log Format) input support for structured logging with custom fields and metadata
- Real-time stream processing with configurable rules for routing, filtering, and enriching log messages
- MongoDB-backed configuration persistence ensuring dashboard and input configurations survive container restarts
- OpenSearch integration providing full-text search across indexed log data with sub-second query response times
- Multi-protocol log ingestion supporting syslog, GELF, and custom TCP/UDP inputs simultaneously
- Built-in alerting system with email notifications based on configurable search conditions and thresholds
- Role-based access control with user authentication and authorization managed through MongoDB
- Stream-based log routing allowing different log types to be processed and stored with separate retention policies

## Use cases

- Application troubleshooting by correlating error logs across microservices and identifying root causes
- Security monitoring and incident response through centralized collection of authentication logs and security events
- Compliance reporting for regulations requiring log retention and audit trails of system access
- Performance monitoring by analyzing application response times and database query logs
- Infrastructure monitoring for server health metrics, network device logs, and cloud service events
- Development environment debugging with real-time log streaming during application testing
- Capacity planning through historical analysis of system resource usage patterns and growth trends

## Prerequisites

- Minimum 4GB RAM available (2GB for OpenSearch, 1GB for MongoDB, 1GB for Graylog)
- Generated GRAYLOG_PASSWORD_SECRET environment variable using openssl or similar cryptographic tool
- SHA256 hash of admin password stored in GRAYLOG_ROOT_PASSWORD_SHA2 environment variable
- Ports 9000, 1514, and 12201 available on the host system for web interface and log ingestion
- Basic understanding of log formats (syslog, JSON) and network protocols for configuring log sources
- At least 10GB available disk space for initial log storage and index data growth

## docker-compose.yml

```yaml
services:
  mongodb:
    image: mongo:6
    container_name: graylog-mongo
    restart: unless-stopped
    volumes:
      - mongo_data:/data/db
    networks:
      - graylog-network

  opensearch:
    image: opensearchproject/opensearch:2
    container_name: graylog-opensearch
    restart: unless-stopped
    environment:
      discovery.type: single-node
      OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
      DISABLE_SECURITY_PLUGIN: "true"
    volumes:
      - opensearch_data:/usr/share/opensearch/data
    networks:
      - graylog-network

  graylog:
    image: graylog/graylog:5.2
    container_name: graylog
    restart: unless-stopped
    environment:
      GRAYLOG_PASSWORD_SECRET: ${GRAYLOG_PASSWORD_SECRET}
      GRAYLOG_ROOT_PASSWORD_SHA2: ${GRAYLOG_ROOT_PASSWORD_SHA2}
      GRAYLOG_HTTP_EXTERNAL_URI: http://localhost:9000/
      GRAYLOG_ELASTICSEARCH_HOSTS: http://opensearch:9200
      GRAYLOG_MONGODB_URI: mongodb://mongodb:27017/graylog
    ports:
      - "9000:9000"
      - "1514:1514"
      - "1514:1514/udp"
      - "12201:12201"
      - "12201:12201/udp"
    depends_on:
      - mongodb
      - opensearch
    networks:
      - graylog-network

volumes:
  mongo_data:
  opensearch_data:

networks:
  graylog-network:
    driver: bridge
```

## .env

```bash
GRAYLOG_PASSWORD_SECRET=somepasswordpepper
GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
```

## Usage notes

- Docs: https://go2docs.graylog.org/
- UI at http://localhost:9000 - login admin with SHA2 password hash
- Generate password hash: echo -n yourpassword | sha256sum
- GELF input on port 12201 (TCP/UDP) - native Graylog format
- Syslog input on port 1514 for traditional log forwarding
- Create Inputs in System > Inputs before logs can be received

## Troubleshooting

- Graylog shows 'Indexer failures' in web interface: Verify OpenSearch container is running and GRAYLOG_ELASTICSEARCH_HOSTS points to correct OpenSearch service name
- Cannot login with admin credentials: Ensure GRAYLOG_ROOT_PASSWORD_SHA2 contains SHA256 hash, not plaintext password, and regenerate hash if necessary
- Logs not appearing in Graylog interface: Create and start appropriate input (GELF, Syslog) in System > Inputs before sending log data
- OpenSearch container exits with OutOfMemoryError: Reduce OPENSEARCH_JAVA_OPTS heap size or increase Docker container memory limits
- MongoDB connection errors in Graylog logs: Check GRAYLOG_MONGODB_URI format and ensure MongoDB service is accessible on port 27017
- Web interface shows 'No active inputs' warning: Configure at least one input source in System > Inputs and ensure it's in 'running' state

