# SeaweedFS Distributed Storage

> Fast distributed object storage system for billions of files.

- Category: Storage & Backup
- Difficulty: advanced
- Components: seaweedfs-master, seaweedfs-volume, seaweedfs-filer, seaweedfs-s3
- Tags: storage, object-storage, seaweedfs, distributed, s3
- Source: https://docker.recipes/storage/seaweedfs-distributed

## Overview

SeaweedFS is a highly scalable distributed object storage system designed to handle billions of files efficiently. Originally developed by Chris Lu, it combines the simplicity of file systems with the scalability of distributed storage, offering both POSIX-compatible file access and S3-compatible object storage APIs. The system achieves exceptional performance by separating metadata management from data storage and using a unique approach that eliminates the need for centralized metadata servers.

This distributed storage stack consists of four essential components working in concert: seaweedfs-master handles cluster coordination and volume management, multiple seaweedfs-volume servers provide the actual data storage capacity, seaweedfs-filer offers POSIX-like file system semantics and acts as a metadata layer, and seaweedfs-s3 provides Amazon S3-compatible API access. The master server maintains the global view of all volumes and their locations, while volume servers store the actual file chunks. The filer enables hierarchical file organization and supports various metadata stores, while the S3 gateway allows existing S3-compatible applications to use SeaweedFS without modification.

This configuration is ideal for organizations requiring petabyte-scale storage with high throughput and low latency, particularly those dealing with large numbers of small to medium-sized files. Unlike traditional distributed file systems that struggle with metadata bottlenecks, SeaweedFS excels in scenarios involving massive file counts while maintaining simplicity in deployment and operation. It's particularly valuable for content delivery networks, backup systems, data lakes, and any application requiring both file system and object storage interfaces.

## Features

- Automatic volume management with configurable size limits and replication policies
- Dual storage interfaces supporting both POSIX file operations and S3-compatible object storage
- Horizontal scalability allowing dynamic addition of volume servers without downtime
- Efficient small file handling through volume-based storage architecture eliminating metadata bottlenecks
- Built-in data deduplication and compression to optimize storage utilization
- Cross-datacenter replication support for geographic distribution and disaster recovery
- Erasure coding options for space-efficient data protection across multiple volumes
- Real-time cluster monitoring and management through web-based administrative interfaces

## Use cases

- Content delivery networks requiring fast access to millions of images, videos, and static assets
- Backup and archival systems handling large volumes of incremental backups and snapshots
- Data lakes storing diverse datasets for analytics workloads with mixed access patterns
- Container image registries needing efficient storage and distribution of Docker images
- Media processing pipelines handling video transcoding and image manipulation workflows
- IoT data ingestion systems collecting and storing sensor data from thousands of devices
- Development environments requiring S3-compatible storage for testing cloud applications locally

## Prerequisites

- Minimum 4GB RAM recommended for master server and 2GB per volume server for optimal performance
- Available ports 9333, 19333 for master services, 8888, 18888 for filer, and 8333 for S3 API
- Understanding of distributed storage concepts including replication, consistency, and volume management
- S3 configuration file (s3.json) properly configured with authentication and bucket settings
- Sufficient disk space across volume servers based on expected data growth and replication factor
- Network connectivity between all nodes with low latency for optimal cluster communication

## docker-compose.yml

```yaml
services:
  master:
    image: chrislusf/seaweedfs:latest
    container_name: seaweedfs-master
    command: master -ip=master -volumeSizeLimitMB=100
    ports:
      - "9333:9333"
      - "19333:19333"
    volumes:
      - master-data:/data
    networks:
      - seaweedfs-network
    restart: unless-stopped

  volume1:
    image: chrislusf/seaweedfs:latest
    container_name: seaweedfs-volume1
    command: volume -mserver="master:9333" -ip=volume1 -port=8080 -max=100
    volumes:
      - volume1-data:/data
    depends_on:
      - master
    networks:
      - seaweedfs-network
    restart: unless-stopped

  volume2:
    image: chrislusf/seaweedfs:latest
    container_name: seaweedfs-volume2
    command: volume -mserver="master:9333" -ip=volume2 -port=8080 -max=100
    volumes:
      - volume2-data:/data
    depends_on:
      - master
    networks:
      - seaweedfs-network
    restart: unless-stopped

  filer:
    image: chrislusf/seaweedfs:latest
    container_name: seaweedfs-filer
    command: filer -master="master:9333" -ip=filer
    ports:
      - "8888:8888"
      - "18888:18888"
    volumes:
      - filer-data:/data
    depends_on:
      - master
      - volume1
      - volume2
    networks:
      - seaweedfs-network
    restart: unless-stopped

  s3:
    image: chrislusf/seaweedfs:latest
    container_name: seaweedfs-s3
    command: s3 -filer="filer:8888" -config=/etc/seaweedfs/s3.json
    ports:
      - "8333:8333"
    volumes:
      - ./s3.json:/etc/seaweedfs/s3.json:ro
    depends_on:
      - filer
    networks:
      - seaweedfs-network
    restart: unless-stopped

volumes:
  master-data:
  volume1-data:
  volume2-data:
  filer-data:

networks:
  seaweedfs-network:
    driver: bridge
```

## .env

```bash
# SeaweedFS
# Create s3.json for S3 credentials:
# {
#   "identities": [
#     {
#       "name": "admin",
#       "credentials": [
#         {"accessKey": "admin", "secretKey": "adminpassword"}
#       ],
#       "actions": ["Admin", "Read", "Write"]
#     }
#   ]
# }
```

## Usage notes

- Master UI at http://localhost:9333
- Filer at http://localhost:8888
- S3 API at http://localhost:8333
- High-performance object storage
- Supports file upload via HTTP

## Troubleshooting

- Master server fails to start with 'bind: address already in use': Check if ports 9333 or 19333 are occupied by other services and modify port mappings if necessary
- Volume servers cannot connect to master: Verify network connectivity between containers and ensure master container is fully started before volume servers attempt connection
- S3 API returns 'filer not accessible' errors: Confirm s3.json configuration file exists and contains correct filer endpoint, and verify filer service is running and accessible
- File uploads fail with 'no writable volumes' error: Check volume server capacity limits and increase -max parameter or add additional volume servers to the cluster
- Filer web interface shows empty directories: Ensure volume servers are properly registered with master and check volume server logs for storage backend issues
- High memory usage on master server: Reduce -volumeSizeLimitMB parameter or distribute load across multiple master servers in production environments

