MinIO Distributed Storage

intermediate

MinIO distributed object storage cluster.

Overview

MinIO is a high-performance, S3-compatible object storage server originally developed to provide cloud-native storage for modern applications. Born from the need for a lightweight yet powerful alternative to Amazon S3, MinIO delivers enterprise-grade object storage with exceptional performance, supporting throughput speeds of multiple GB/s while maintaining full compatibility with the S3 API. This makes it an ideal choice for organizations seeking to reduce cloud storage costs or maintain complete control over their data infrastructure. This distributed MinIO cluster configuration deploys four MinIO server instances behind an Nginx load balancer, creating a fault-tolerant object storage system with automatic erasure coding. The four-node setup provides optimal data protection through MinIO's distributed erasure coding algorithm, which can tolerate up to two node failures while maintaining data availability. Nginx acts as the frontend proxy, distributing API requests across all healthy MinIO nodes and providing a single endpoint for both S3 API calls and web console access. This stack is perfect for organizations running data-intensive applications, AI/ML workloads, or those requiring a self-hosted S3 replacement with high availability guarantees. The distributed architecture ensures that your object storage remains operational even during node maintenance or unexpected failures, while the S3 compatibility allows existing applications to work without modification. Development teams benefit from having a production-like S3 environment for testing, while enterprises gain the ability to store petabytes of data without vendor lock-in or egress fees.

Key Features

100% Amazon S3 API compatibility for drop-in replacement of S3 services
Distributed erasure coding across four nodes with tolerance for up to two node failures
High-performance object storage delivering multi-GB/s throughput for large file operations
Built-in web console for bucket management, user administration, and monitoring
Automatic data protection with configurable retention policies and object versioning
Enterprise-grade encryption at rest and in transit using AES-256 and TLS
Load-balanced access through Nginx proxy for optimal request distribution
Active health checking and automatic failover for uninterrupted service availability

Common Use Cases

1Self-hosted S3 replacement for organizations wanting to eliminate cloud storage costs and data egress fees
2AI/ML data lakes requiring high-throughput storage for training datasets and model artifacts
3Media and content delivery platforms needing scalable storage for videos, images, and static assets
4Backup and archival systems for enterprises with long-term data retention requirements
5Development and testing environments requiring S3-compatible storage without cloud dependencies
6Edge computing deployments where data sovereignty and local storage control are critical
7Kubernetes-native applications requiring persistent object storage with S3 API compatibility

Prerequisites

Minimum 8GB RAM (2GB per MinIO node) with 16GB+ recommended for production workloads
Available ports 9000 and 9001 on the host system for S3 API and web console access
Nginx configuration file (nginx.conf) created in the same directory as docker-compose.yml
At least 100GB of available disk space per node for meaningful object storage capacity
Understanding of S3 API concepts including buckets, objects, and access key management
Basic knowledge of load balancers and distributed storage concepts for troubleshooting

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

1services: 
2  minio1: 
3    image: minio/minio:latest
4    container_name: minio1
5    restart: unless-stopped
6    command: server --console-address ":9001" http://minio{1...4}/data
7    environment: 
8      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
9      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
10    volumes: 
11      - minio1_data:/data
12    networks: 
13      - minio-network
14
15  minio2: 
16    image: minio/minio:latest
17    container_name: minio2
18    restart: unless-stopped
19    command: server --console-address ":9001" http://minio{1...4}/data
20    environment: 
21      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
22      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
23    volumes: 
24      - minio2_data:/data
25    networks: 
26      - minio-network
27
28  minio3: 
29    image: minio/minio:latest
30    container_name: minio3
31    restart: unless-stopped
32    command: server --console-address ":9001" http://minio{1...4}/data
33    environment: 
34      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
35      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
36    volumes: 
37      - minio3_data:/data
38    networks: 
39      - minio-network
40
41  minio4: 
42    image: minio/minio:latest
43    container_name: minio4
44    restart: unless-stopped
45    command: server --console-address ":9001" http://minio{1...4}/data
46    environment: 
47      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
48      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
49    volumes: 
50      - minio4_data:/data
51    networks: 
52      - minio-network
53
54  nginx: 
55    image: nginx:alpine
56    container_name: minio-lb
57    restart: unless-stopped
58    ports: 
59      - "${MINIO_PORT:-9000}:9000"
60      - "${CONSOLE_PORT:-9001}:9001"
61    volumes: 
62      - ./nginx.conf:/etc/nginx/nginx.conf:ro
63    depends_on: 
64      - minio1
65      - minio2
66      - minio3
67      - minio4
68    networks: 
69      - minio-network
70
71volumes: 
72  minio1_data: 
73  minio2_data: 
74  minio3_data: 
75  minio4_data: 
76
77networks: 
78  minio-network: 
79    driver: bridge

.env Template

.env

1# MinIO Distributed
2MINIO_PORT=9000
3CONSOLE_PORT=9001
4MINIO_USER=minioadmin
5MINIO_PASSWORD=minioadmin

Usage Notes

1API at http://localhost:9000
2Console at http://localhost:9001
3Create nginx.conf for load balancing
4Erasure coded for data protection

Individual Services(5 services)

Copy individual services to mix and match with your existing compose files.

minio1

minio1:
  image: minio/minio:latest
  container_name: minio1
  restart: unless-stopped
  command: server --console-address ":9001" http://minio{1...4}/data
  environment:
    MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
    MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
  volumes:
    - minio1_data:/data
  networks:
    - minio-network

minio2

minio2:
  image: minio/minio:latest
  container_name: minio2
  restart: unless-stopped
  command: server --console-address ":9001" http://minio{1...4}/data
  environment:
    MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
    MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
  volumes:
    - minio2_data:/data
  networks:
    - minio-network

minio3

minio3:
  image: minio/minio:latest
  container_name: minio3
  restart: unless-stopped
  command: server --console-address ":9001" http://minio{1...4}/data
  environment:
    MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
    MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
  volumes:
    - minio3_data:/data
  networks:
    - minio-network

minio4

minio4:
  image: minio/minio:latest
  container_name: minio4
  restart: unless-stopped
  command: server --console-address ":9001" http://minio{1...4}/data
  environment:
    MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
    MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
  volumes:
    - minio4_data:/data
  networks:
    - minio-network

nginx

nginx:
  image: nginx:alpine
  container_name: minio-lb
  restart: unless-stopped
  ports:
    - ${MINIO_PORT:-9000}:9000
    - ${CONSOLE_PORT:-9001}:9001
  volumes:
    - ./nginx.conf:/etc/nginx/nginx.conf:ro
  depends_on:
    - minio1
    - minio2
    - minio3
    - minio4
  networks:
    - minio-network

Quick Start

terminal

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  minio1:
5    image: minio/minio:latest
6    container_name: minio1
7    restart: unless-stopped
8    command: server --console-address ":9001" http://minio{1...4}/data
9    environment:
10      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
11      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
12    volumes:
13      - minio1_data:/data
14    networks:
15      - minio-network
16
17  minio2:
18    image: minio/minio:latest
19    container_name: minio2
20    restart: unless-stopped
21    command: server --console-address ":9001" http://minio{1...4}/data
22    environment:
23      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
24      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
25    volumes:
26      - minio2_data:/data
27    networks:
28      - minio-network
29
30  minio3:
31    image: minio/minio:latest
32    container_name: minio3
33    restart: unless-stopped
34    command: server --console-address ":9001" http://minio{1...4}/data
35    environment:
36      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
37      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
38    volumes:
39      - minio3_data:/data
40    networks:
41      - minio-network
42
43  minio4:
44    image: minio/minio:latest
45    container_name: minio4
46    restart: unless-stopped
47    command: server --console-address ":9001" http://minio{1...4}/data
48    environment:
49      MINIO_ROOT_USER: ${MINIO_USER:-minioadmin}
50      MINIO_ROOT_PASSWORD: ${MINIO_PASSWORD:-minioadmin}
51    volumes:
52      - minio4_data:/data
53    networks:
54      - minio-network
55
56  nginx:
57    image: nginx:alpine
58    container_name: minio-lb
59    restart: unless-stopped
60    ports:
61      - "${MINIO_PORT:-9000}:9000"
62      - "${CONSOLE_PORT:-9001}:9001"
63    volumes:
64      - ./nginx.conf:/etc/nginx/nginx.conf:ro
65    depends_on:
66      - minio1
67      - minio2
68      - minio3
69      - minio4
70    networks:
71      - minio-network
72
73volumes:
74  minio1_data:
75  minio2_data:
76  minio3_data:
77  minio4_data:
78
79networks:
80  minio-network:
81    driver: bridge
82EOF
83
84# 2. Create the .env file
85cat > .env << 'EOF'
86# MinIO Distributed
87MINIO_PORT=9000
88CONSOLE_PORT=9001
89MINIO_USER=minioadmin
90MINIO_PASSWORD=minioadmin
91EOF
92
93# 3. Start the services
94docker compose up -d
95
96# 4. View logs
97docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal

1curl -fsSL https://docker.recipes/api/recipes/minio-distributed/run | bash

Troubleshooting

MinIO nodes fail to start with 'Invalid command line arguments': Ensure all four container names (minio1-4) are resolvable within the Docker network and the server command syntax matches the node count
Web console shows 'MinIO Console is loaded but API is not reachable': Verify that Nginx is properly forwarding requests and all MinIO containers are healthy using docker-compose ps
S3 operations fail with 'NoSuchBucket' after cluster restart: MinIO distributed mode requires consistent node ordering - ensure containers always start with the same names and network configuration
High memory usage and performance degradation: MinIO caches frequently accessed objects in RAM - monitor memory usage and consider increasing container memory limits or reducing concurrent operations
Nginx returns 502 Bad Gateway errors: Check if all four MinIO nodes are running and accessible on port 9000 within the Docker network using docker exec commands
Data inconsistency after node failure: Allow MinIO's healing process to complete automatically - this can take time depending on data volume and may require manual triggering via mc admin heal command

Community Notes

Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

minio

Related Recipes

Ad Space

Shortcuts: C CopyF FavoriteD Download