# Grafana Mimir

> Horizontally scalable long-term storage for Prometheus metrics.

- Category: Monitoring & Observability
- Difficulty: advanced
- Components: mimir, minio, grafana
- Tags: mimir, grafana, prometheus, metrics, scalable
- Source: https://docker.recipes/monitoring/mimir

## Overview

Grafana Mimir is a horizontally scalable, highly available time series database specifically designed for long-term storage of Prometheus metrics. Born from Grafana Labs' experience with Cortex, Mimir addresses the limitations of Prometheus' local storage by providing unlimited retention, multi-tenancy, and the ability to scale storage and query performance independently. Unlike Prometheus' built-in storage which is limited by single-node constraints, Mimir can store years of metrics data while maintaining fast query performance across massive datasets.

This stack combines Mimir with MinIO as the S3-compatible object storage backend and Grafana for visualization, creating a complete metrics platform that can handle enterprise-scale monitoring workloads. MinIO serves as the durable storage layer where Mimir stores its blocks and metadata, eliminating the need for expensive cloud storage while providing the same S3 API compatibility. The integration allows Mimir to leverage MinIO's high-performance object storage for both recent and historical metric data, while Grafana connects directly to Mimir's Prometheus-compatible query API.

This configuration is ideal for organizations that have outgrown single-node Prometheus deployments or need to retain metrics for extended periods for capacity planning, compliance, or historical analysis. DevOps teams managing multiple Prometheus instances can consolidate their metrics into Mimir for centralized querying, while platform engineers can provide a multi-tenant metrics service to multiple teams. The combination provides enterprise-grade metrics storage without vendor lock-in or cloud storage costs.

## Features

- Prometheus remote_write compatibility for drop-in replacement of Prometheus storage
- Multi-tenancy support with per-tenant query isolation and rate limiting
- Horizontal scaling with independent scaling of ingestion, querying, and storage components
- S3-compatible object storage backend via MinIO for cost-effective long-term retention
- PromQL query federation across multiple Prometheus instances and tenants
- Grafana native integration with Prometheus datasource compatibility
- Block-based storage format optimized for time series data compression and query performance
- Built-in compaction and downsampling for storage efficiency over time

## Use cases

- Consolidating metrics from multiple Prometheus instances into centralized long-term storage
- Multi-tenant SaaS platforms providing isolated metrics storage per customer or team
- Financial services requiring extended metrics retention for regulatory compliance
- Large-scale Kubernetes environments exceeding single Prometheus node capacity
- Hybrid cloud deployments needing unified metrics storage across multiple data centers
- Cost optimization by replacing expensive cloud object storage with self-hosted MinIO
- Historical capacity planning and trend analysis across years of infrastructure metrics

## Prerequisites

- Minimum 4GB RAM (2GB for Mimir, 2GB for MinIO, 512MB for Grafana)
- Available ports 3000, 8080, 9000, and 9001 for service access
- Mimir configuration file at ./mimir/mimir.yaml with MinIO backend settings
- Environment variables MINIO_PASSWORD and GRAFANA_PASSWORD configured
- Understanding of PromQL and Prometheus remote_write configuration
- Basic knowledge of S3 API and object storage concepts for MinIO management

## docker-compose.yml

```yaml
services:
  minio:
    image: minio/minio:latest
    container_name: mimir-minio
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: mimir
      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD}
    volumes:
      - minio_data:/data
    ports:
      - "9000:9000"
      - "9001:9001"
    networks:
      - mimir-network

  mimir:
    image: grafana/mimir:latest
    container_name: mimir
    restart: unless-stopped
    command:
      - --config.file=/etc/mimir/mimir.yaml
    volumes:
      - ./mimir/mimir.yaml:/etc/mimir/mimir.yaml:ro
    ports:
      - "8080:8080"
    depends_on:
      - minio
    networks:
      - mimir-network

  grafana:
    image: grafana/grafana:latest
    container_name: grafana
    restart: unless-stopped
    environment:
      GF_SECURITY_ADMIN_PASSWORD: ${GRAFANA_PASSWORD}
    ports:
      - "3000:3000"
    depends_on:
      - mimir
    networks:
      - mimir-network

volumes:
  minio_data:

networks:
  mimir-network:
    driver: bridge
```

## .env

```bash
MINIO_PASSWORD=changeme123
GRAFANA_PASSWORD=admin
```

## Usage notes

- Docs: https://grafana.com/docs/mimir/latest/
- Prometheus remote_write endpoint: http://localhost:8080/api/v1/push
- Create mimir/mimir.yaml config before starting
- MinIO provides S3-compatible storage backend
- Horizontally scalable - add more Mimir instances for HA
- Grafana datasource: add as Prometheus type with Mimir URL

## Troubleshooting

- Mimir fails to start with 'no such file or directory': Create ./mimir/mimir.yaml configuration file with proper MinIO backend settings before starting containers
- MinIO connection refused errors in Mimir logs: Verify minio service is fully started and accessible on port 9000 before Mimir attempts connection
- Grafana shows 'Bad Gateway' when querying Mimir: Check that Mimir container is running and accessible on port 8080, verify network connectivity between containers
- Out of memory errors during metric ingestion: Increase Docker memory limits and ensure adequate memory allocation for Mimir's ingestion components
- Slow query performance on historical data: Verify MinIO storage performance and check Mimir compaction settings for proper block organization
- Authentication failed errors: Ensure MINIO_ROOT_USER matches credentials configured in Mimir's S3 backend configuration

