Milvus

intermediate

Cloud-native vector database for AI workloads.

Overview

Milvus is an open-source vector database designed for AI applications requiring high-performance similarity search across massive datasets. Originally developed by Zilliz in 2019, Milvus has become the go-to solution for organizations needing to search through billions of vectors with millisecond latency, supporting advanced indexing algorithms like HNSW, IVF, and DiskANN for trillion-scale deployments. This stack combines Milvus with etcd for distributed coordination and MinIO for object storage, creating a complete vector database infrastructure. etcd manages Milvus's metadata and cluster state with strong consistency guarantees, while MinIO provides S3-compatible storage for vector data, index files, and binary logs. Together, they enable Milvus to scale horizontally while maintaining data durability and consistency across distributed deployments. This configuration is ideal for AI engineers, data scientists, and organizations building production-grade similarity search applications. The combination delivers enterprise-grade reliability for computer vision pipelines, recommendation engines, and semantic search systems that need to process millions of embeddings daily. Companies migrating from managed services like Pinecone or building self-hosted alternatives to Weaviate will find this stack provides comparable performance with full infrastructure control.

Key Features

Trillion-scale vector indexing with HNSW, IVF, and DiskANN algorithms for sub-millisecond search
Hybrid scalar and vector search enabling filtered similarity queries across metadata
Time travel queries allowing point-in-time data recovery and version control
GPU acceleration support for index building and search operations
Multi-tenancy with collection-level isolation and role-based access control
S3-compatible object storage through MinIO for cost-effective data persistence
Distributed metadata management via etcd with automatic failover capabilities
Native support for multiple vector types including float, binary, and sparse vectors

Common Use Cases

1Computer vision applications requiring reverse image search across millions of photos
2Recommendation systems matching user preferences against product embeddings
3Semantic search engines processing natural language queries against document collections
4Drug discovery platforms comparing molecular structures and chemical compounds
5Fraud detection systems identifying anomalous patterns in financial transactions
6Content moderation tools flagging similar images, videos, or text across platforms
7Real-time personalization engines matching user behavior to content embeddings

Prerequisites

Minimum 8GB RAM for production workloads (2GB absolute minimum for testing)
Available ports 19530 (gRPC), 9091 (metrics), 2379 (etcd), 9000/9001 (MinIO)
Understanding of vector embeddings and similarity search concepts
Python development environment for pymilvus client integration
Basic knowledge of gRPC APIs and protocol buffer schemas
Storage planning for vector data growth (can scale to terabytes rapidly)

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

1services: 
2  milvus: 
3    image: milvusdb/milvus:latest
4    container_name: milvus
5    command: milvus run standalone
6    environment: 
7      ETCD_ENDPOINTS: etcd:2379
8      MINIO_ADDRESS: minio:9000
9    volumes: 
10      - milvus_data:/var/lib/milvus
11    ports: 
12      - "19530:19530"
13      - "9091:9091"
14    depends_on: 
15      - etcd
16      - minio
17    networks: 
18      - milvus
19
20  etcd: 
21    image: quay.io/coreos/etcd:v3.5.5
22    container_name: milvus-etcd
23    command: etcd -advertise-client-urls=http://etcd:2379 -listen-client-urls=http://0.0.0.0:2379
24    volumes: 
25      - etcd_data:/etcd
26    networks: 
27      - milvus
28
29  minio: 
30    image: minio/minio:latest
31    container_name: milvus-minio
32    command: server /data --console-address ":9001"
33    environment: 
34      MINIO_ROOT_USER: minioadmin
35      MINIO_ROOT_PASSWORD: minioadmin
36    volumes: 
37      - minio_data:/data
38    networks: 
39      - milvus
40
41volumes: 
42  milvus_data: 
43  etcd_data: 
44  minio_data: 
45
46networks: 
47  milvus: 
48    driver: bridge

.env Template

.env

1# Default configuration

Usage Notes

1Docs: https://milvus.io/docs
2gRPC API at localhost:19530 - primary interface
3Management/metrics at http://localhost:9091/metrics
4Python client: pip install pymilvus
5Create collection with schema defining vector dimensions
6Supports IVF, HNSW, DiskANN indexes for billion-scale search

Individual Services(3 services)

Copy individual services to mix and match with your existing compose files.

milvus

milvus:
  image: milvusdb/milvus:latest
  container_name: milvus
  command: milvus run standalone
  environment:
    ETCD_ENDPOINTS: etcd:2379
    MINIO_ADDRESS: minio:9000
  volumes:
    - milvus_data:/var/lib/milvus
  ports:
    - "19530:19530"
    - "9091:9091"
  depends_on:
    - etcd
    - minio
  networks:
    - milvus

etcd

etcd:
  image: quay.io/coreos/etcd:v3.5.5
  container_name: milvus-etcd
  command: etcd -advertise-client-urls=http://etcd:2379 -listen-client-urls=http://0.0.0.0:2379
  volumes:
    - etcd_data:/etcd
  networks:
    - milvus

minio

minio:
  image: minio/minio:latest
  container_name: milvus-minio
  command: server /data --console-address ":9001"
  environment:
    MINIO_ROOT_USER: minioadmin
    MINIO_ROOT_PASSWORD: minioadmin
  volumes:
    - minio_data:/data
  networks:
    - milvus

Quick Start

terminal

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  milvus:
5    image: milvusdb/milvus:latest
6    container_name: milvus
7    command: milvus run standalone
8    environment:
9      ETCD_ENDPOINTS: etcd:2379
10      MINIO_ADDRESS: minio:9000
11    volumes:
12      - milvus_data:/var/lib/milvus
13    ports:
14      - "19530:19530"
15      - "9091:9091"
16    depends_on:
17      - etcd
18      - minio
19    networks:
20      - milvus
21
22  etcd:
23    image: quay.io/coreos/etcd:v3.5.5
24    container_name: milvus-etcd
25    command: etcd -advertise-client-urls=http://etcd:2379 -listen-client-urls=http://0.0.0.0:2379
26    volumes:
27      - etcd_data:/etcd
28    networks:
29      - milvus
30
31  minio:
32    image: minio/minio:latest
33    container_name: milvus-minio
34    command: server /data --console-address ":9001"
35    environment:
36      MINIO_ROOT_USER: minioadmin
37      MINIO_ROOT_PASSWORD: minioadmin
38    volumes:
39      - minio_data:/data
40    networks:
41      - milvus
42
43volumes:
44  milvus_data:
45  etcd_data:
46  minio_data:
47
48networks:
49  milvus:
50    driver: bridge
51EOF
52
53# 2. Create the .env file
54cat > .env << 'EOF'
55# Default configuration
56EOF
57
58# 3. Start the services
59docker compose up -d
60
61# 4. View logs
62docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal

1curl -fsSL https://docker.recipes/api/recipes/milvus/run | bash

Troubleshooting

Connection refused on port 19530: Verify etcd and MinIO are healthy before Milvus starts
Out of memory errors during indexing: Increase container memory limits and reduce index_build_params
Slow search performance: Switch from IVF_FLAT to HNSW index type for better query speed
MinIO access denied errors: Ensure MINIO_ROOT_USER/PASSWORD match Milvus configuration
etcd cluster unhealthy: Check etcd logs and ensure sufficient disk space for metadata storage
Collection loading failures: Verify vector dimensions match between schema and data insertion

Community Notes

Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

milvusetcdminio

Related Recipes

Ad Space

Shortcuts: C CopyF FavoriteD Download