# MinIO Distributed Storage

> MinIO distributed object storage cluster.

- Category: Storage & Backup
- Difficulty: intermediate
- Components: minio
- Tags: minio, s3, object-storage, distributed, cluster
- Source: https://docker.recipes/storage/minio-distributed

## Overview

MinIO is a high-performance, S3-compatible object storage server originally developed to provide cloud-native storage for modern applications. Born from the need for a lightweight yet powerful alternative to Amazon S3, MinIO delivers enterprise-grade object storage with exceptional performance, supporting throughput speeds of multiple GB/s while maintaining full compatibility with the S3 API. This makes it an ideal choice for organizations seeking to reduce cloud storage costs or maintain complete control over their data infrastructure.

This distributed MinIO cluster configuration deploys four MinIO server instances behind an Nginx load balancer, creating a fault-tolerant object storage system with automatic erasure coding. The four-node setup provides optimal data protection through MinIO's distributed erasure coding algorithm, which can tolerate up to two node failures while maintaining data availability. Nginx acts as the frontend proxy, distributing API requests across all healthy MinIO nodes and providing a single endpoint for both S3 API calls and web console access.

This stack is perfect for organizations running data-intensive applications, AI/ML workloads, or those requiring a self-hosted S3 replacement with high availability guarantees. The distributed architecture ensures that your object storage remains operational even during node maintenance or unexpected failures, while the S3 compatibility allows existing applications to work without modification. Development teams benefit from having a production-like S3 environment for testing, while enterprises gain the ability to store petabytes of data without vendor lock-in or egress fees.

## Features

- 100% Amazon S3 API compatibility for drop-in replacement of S3 services
- Distributed erasure coding across four nodes with tolerance for up to two node failures
- High-performance object storage delivering multi-GB/s throughput for large file operations
- Built-in web console for bucket management, user administration, and monitoring
- Automatic data protection with configurable retention policies and object versioning
- Enterprise-grade encryption at rest and in transit using AES-256 and TLS
- Load-balanced access through Nginx proxy for optimal request distribution
- Active health checking and automatic failover for uninterrupted service availability

## Use cases

- Self-hosted S3 replacement for organizations wanting to eliminate cloud storage costs and data egress fees
- AI/ML data lakes requiring high-throughput storage for training datasets and model artifacts
- Media and content delivery platforms needing scalable storage for videos, images, and static assets
- Backup and archival systems for enterprises with long-term data retention requirements
- Development and testing environments requiring S3-compatible storage without cloud dependencies
- Edge computing deployments where data sovereignty and local storage control are critical
- Kubernetes-native applications requiring persistent object storage with S3 API compatibility

## Prerequisites

- Minimum 8GB RAM (2GB per MinIO node) with 16GB+ recommended for production workloads
- Available ports 9000 and 9001 on the host system for S3 API and web console access
- Nginx configuration file (nginx.conf) created in the same directory as docker-compose.yml
- At least 100GB of available disk space per node for meaningful object storage capacity
- Understanding of S3 API concepts including buckets, objects, and access key management
- Basic knowledge of load balancers and distributed storage concepts for troubleshooting

## docker-compose.yml

```yaml
services:
  minio1:
    image: minio/minio:latest
    container_name: minio1
    restart: unless-stopped
    command: server --console-address ":9001" http://minio{1...4}/data
    environment:
      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
    volumes:
      - minio1_data:/data
    networks:
      - minio-network

  minio2:
    image: minio/minio:latest
    container_name: minio2
    restart: unless-stopped
    command: server --console-address ":9001" http://minio{1...4}/data
    environment:
      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
    volumes:
      - minio2_data:/data
    networks:
      - minio-network

  minio3:
    image: minio/minio:latest
    container_name: minio3
    restart: unless-stopped
    command: server --console-address ":9001" http://minio{1...4}/data
    environment:
      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
    volumes:
      - minio3_data:/data
    networks:
      - minio-network

  minio4:
    image: minio/minio:latest
    container_name: minio4
    restart: unless-stopped
    command: server --console-address ":9001" http://minio{1...4}/data
    environment:
      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
    volumes:
      - minio4_data:/data
    networks:
      - minio-network

  nginx:
    image: nginx:alpine
    container_name: minio-lb
    restart: unless-stopped
    ports:
      - "${MINIO_PORT:-9000}:9000"
      - "${CONSOLE_PORT:-9001}:9001"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - minio1
      - minio2
      - minio3
      - minio4
    networks:
      - minio-network

volumes:
  minio1_data:
  minio2_data:
  minio3_data:
  minio4_data:

networks:
  minio-network:
    driver: bridge
```

## .env

```bash
# MinIO Distributed
MINIO_PORT=9000
CONSOLE_PORT=9001
MINIO_USER=minioadmin
MINIO_PASSWORD=minioadmin
```

## Usage notes

- API at http://localhost:9000
- Console at http://localhost:9001
- Create nginx.conf for load balancing
- Erasure coded for data protection

## Troubleshooting

- MinIO nodes fail to start with 'Invalid command line arguments': Ensure all four container names (minio1-4) are resolvable within the Docker network and the server command syntax matches the node count
- Web console shows 'MinIO Console is loaded but API is not reachable': Verify that Nginx is properly forwarding requests and all MinIO containers are healthy using docker-compose ps
- S3 operations fail with 'NoSuchBucket' after cluster restart: MinIO distributed mode requires consistent node ordering - ensure containers always start with the same names and network configuration
- High memory usage and performance degradation: MinIO caches frequently accessed objects in RAM - monitor memory usage and consider increasing container memory limits or reducing concurrent operations
- Nginx returns 502 Bad Gateway errors: Check if all four MinIO nodes are running and accessible on port 9000 within the Docker network using docker exec commands
- Data inconsistency after node failure: Allow MinIO's healing process to complete automatically - this can take time depending on data volume and may require manual triggering via mc admin heal command

