# Milvus

> Cloud-native vector database for AI workloads.

- Category: AI & Machine Learning
- Difficulty: intermediate
- Components: milvus, etcd, minio
- Tags: milvus, vector-db, embeddings, similarity
- Source: https://docker.recipes/ai-ml/milvus

## Overview

Milvus is an open-source vector database designed for AI applications requiring high-performance similarity search across massive datasets. Originally developed by Zilliz in 2019, Milvus has become the go-to solution for organizations needing to search through billions of vectors with millisecond latency, supporting advanced indexing algorithms like HNSW, IVF, and DiskANN for trillion-scale deployments.

This stack combines Milvus with etcd for distributed coordination and MinIO for object storage, creating a complete vector database infrastructure. etcd manages Milvus's metadata and cluster state with strong consistency guarantees, while MinIO provides S3-compatible storage for vector data, index files, and binary logs. Together, they enable Milvus to scale horizontally while maintaining data durability and consistency across distributed deployments.

This configuration is ideal for AI engineers, data scientists, and organizations building production-grade similarity search applications. The combination delivers enterprise-grade reliability for computer vision pipelines, recommendation engines, and semantic search systems that need to process millions of embeddings daily. Companies migrating from managed services like Pinecone or building self-hosted alternatives to Weaviate will find this stack provides comparable performance with full infrastructure control.

## Features

- Trillion-scale vector indexing with HNSW, IVF, and DiskANN algorithms for sub-millisecond search
- Hybrid scalar and vector search enabling filtered similarity queries across metadata
- Time travel queries allowing point-in-time data recovery and version control
- GPU acceleration support for index building and search operations
- Multi-tenancy with collection-level isolation and role-based access control
- S3-compatible object storage through MinIO for cost-effective data persistence
- Distributed metadata management via etcd with automatic failover capabilities
- Native support for multiple vector types including float, binary, and sparse vectors

## Use cases

- Computer vision applications requiring reverse image search across millions of photos
- Recommendation systems matching user preferences against product embeddings
- Semantic search engines processing natural language queries against document collections
- Drug discovery platforms comparing molecular structures and chemical compounds
- Fraud detection systems identifying anomalous patterns in financial transactions
- Content moderation tools flagging similar images, videos, or text across platforms
- Real-time personalization engines matching user behavior to content embeddings

## Prerequisites

- Minimum 8GB RAM for production workloads (2GB absolute minimum for testing)
- Available ports 19530 (gRPC), 9091 (metrics), 2379 (etcd), 9000/9001 (MinIO)
- Understanding of vector embeddings and similarity search concepts
- Python development environment for pymilvus client integration
- Basic knowledge of gRPC APIs and protocol buffer schemas
- Storage planning for vector data growth (can scale to terabytes rapidly)

## docker-compose.yml

```yaml
services:
  milvus:
    image: milvusdb/milvus:latest
    container_name: milvus
    command: milvus run standalone
    environment:
      ETCD_ENDPOINTS: etcd:2379
      MINIO_ADDRESS: minio:9000
    volumes:
      - milvus_data:/var/lib/milvus
    ports:
      - "19530:19530"
      - "9091:9091"
    depends_on:
      - etcd
      - minio
    networks:
      - milvus

  etcd:
    image: quay.io/coreos/etcd:v3.5.5
    container_name: milvus-etcd
    command: etcd -advertise-client-urls=http://etcd:2379 -listen-client-urls=http://0.0.0.0:2379
    volumes:
      - etcd_data:/etcd
    networks:
      - milvus

  minio:
    image: minio/minio:latest
    container_name: milvus-minio
    command: server /data --console-address ":9001"
    environment:
      MINIO_ROOT_USER: minioadmin
      MINIO_ROOT_PASSWORD: minioadmin
    volumes:
      - minio_data:/data
    networks:
      - milvus

volumes:
  milvus_data:
  etcd_data:
  minio_data:

networks:
  milvus:
    driver: bridge
```

## .env

```bash
# Default configuration
```

## Usage notes

- Docs: https://milvus.io/docs
- gRPC API at localhost:19530 - primary interface
- Management/metrics at http://localhost:9091/metrics
- Python client: pip install pymilvus
- Create collection with schema defining vector dimensions
- Supports IVF, HNSW, DiskANN indexes for billion-scale search

## Troubleshooting

- Connection refused on port 19530: Verify etcd and MinIO are healthy before Milvus starts
- Out of memory errors during indexing: Increase container memory limits and reduce index_build_params
- Slow search performance: Switch from IVF_FLAT to HNSW index type for better query speed
- MinIO access denied errors: Ensure MINIO_ROOT_USER/PASSWORD match Milvus configuration
- etcd cluster unhealthy: Check etcd logs and ensure sufficient disk space for metadata storage
- Collection loading failures: Verify vector dimensions match between schema and data insertion

