# TiDB Distributed Database Cluster

> MySQL-compatible distributed database with TiKV storage and PD cluster management.

- Category: Database Stacks
- Difficulty: advanced
- Components: tidb, tikv, pd, prometheus, grafana
- Tags: tidb, distributed, mysql, tikv, newsql
- Source: https://docker.recipes/databases/tidb-cluster

## Overview

TiDB is a distributed, MySQL-compatible database developed by PingCAP that implements the NewSQL paradigm, combining ACID transactions with horizontal scalability. This open-source database separates compute from storage using TiKV (a distributed key-value store) for data persistence and Placement Driver (PD) for cluster coordination and metadata management. TiDB was designed to handle massive datasets that exceed single-machine capacity while maintaining strong consistency and ACID compliance. This distributed architecture enables TiDB clusters to scale from gigabytes to petabytes while supporting real-time analytics through its columnar storage engine TiFlash. The cluster operates with TiDB servers handling SQL processing and client connections, multiple TiKV nodes providing distributed transactional storage with Raft consensus, and PD managing cluster topology, load balancing, and distributed transactions. Prometheus collects comprehensive metrics from all TiDB components, while Grafana provides specialized dashboards for monitoring cluster health, query performance, and storage utilization. Organizations running high-traffic applications requiring both transactional consistency and analytical capabilities benefit from this stack, particularly those experiencing MySQL scaling limitations or needing real-time HTAP workloads. The combination delivers enterprise-grade distributed database capabilities with comprehensive observability, making it ideal for financial services, e-commerce platforms, and SaaS applications requiring consistent performance at scale.

## Features

- MySQL protocol compatibility enabling existing applications to connect without code changes
- Horizontal scaling with automatic data sharding across TiKV nodes using Region-based distribution
- Strong consistency with distributed ACID transactions using Two-Phase Commit protocol
- Real-time analytics through TiFlash columnar storage engine for HTAP workloads
- Automatic failover and self-healing with Raft consensus algorithm in TiKV clusters
- Multi-version concurrency control (MVCC) for non-blocking reads and snapshot isolation
- Online DDL operations without downtime using TiDB's distributed schema change algorithm
- Comprehensive monitoring with 200+ metrics exported to Prometheus and pre-built Grafana dashboards

## Use cases

- E-commerce platforms requiring real-time inventory management with analytics on transaction patterns
- Financial services needing ACID compliance for payments with real-time fraud detection analytics
- Gaming companies tracking player events and running live leaderboards with high write throughput
- SaaS applications experiencing MySQL scaling bottlenecks requiring transparent horizontal scaling
- IoT platforms ingesting sensor data while running real-time analytics on device performance
- Multi-tenant applications needing predictable performance isolation across customer workloads
- Data warehouse modernization projects requiring both transactional and analytical workloads

## Prerequisites

- Minimum 8GB RAM across all nodes (2GB for TiDB, 4GB for TiKV nodes, 1GB each for PD/monitoring)
- Understanding of distributed systems concepts including Raft consensus and data sharding
- Familiarity with MySQL query syntax and database administration concepts
- Network configuration knowledge for multi-node cluster communication and port management
- Basic Prometheus and Grafana experience for monitoring distributed database metrics
- Docker environment with at least 20GB available storage for persistent data volumes

## docker-compose.yml

```yaml
services:
  pd0:
    image: pingcap/pd:latest
    ports:
      - "2379:2379"
    command:
      - --name=pd0
      - --client-urls=http://0.0.0.0:2379
      - --peer-urls=http://0.0.0.0:2380
      - --advertise-client-urls=http://pd0:2379
      - --advertise-peer-urls=http://pd0:2380
      - --initial-cluster=pd0=http://pd0:2380
      - --data-dir=/data
    volumes:
      - pd0_data:/data
    networks:
      - tidb_net

  tikv0:
    image: pingcap/tikv:latest
    command:
      - --addr=0.0.0.0:20160
      - --advertise-addr=tikv0:20160
      - --data-dir=/data
      - --pd=pd0:2379
    volumes:
      - tikv0_data:/data
    depends_on:
      - pd0
    networks:
      - tidb_net

  tikv1:
    image: pingcap/tikv:latest
    command:
      - --addr=0.0.0.0:20160
      - --advertise-addr=tikv1:20160
      - --data-dir=/data
      - --pd=pd0:2379
    volumes:
      - tikv1_data:/data
    depends_on:
      - pd0
    networks:
      - tidb_net

  tikv2:
    image: pingcap/tikv:latest
    command:
      - --addr=0.0.0.0:20160
      - --advertise-addr=tikv2:20160
      - --data-dir=/data
      - --pd=pd0:2379
    volumes:
      - tikv2_data:/data
    depends_on:
      - pd0
    networks:
      - tidb_net

  tidb:
    image: pingcap/tidb:latest
    ports:
      - "4000:4000"
      - "10080:10080"
    command:
      - --store=tikv
      - --path=pd0:2379
    depends_on:
      - tikv0
      - tikv1
      - tikv2
    networks:
      - tidb_net

  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    networks:
      - tidb_net

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
    volumes:
      - grafana_data:/var/lib/grafana
    networks:
      - tidb_net

volumes:
  pd0_data:
  tikv0_data:
  tikv1_data:
  tikv2_data:
  prometheus_data:
  grafana_data:

networks:
  tidb_net:
```

## .env

```bash
# TiDB Cluster
GRAFANA_PASSWORD=secure_grafana_password

# MySQL-compatible at localhost:4000
# Status at http://localhost:10080
```

## Usage notes

- MySQL-compatible connection at port 4000
- Use standard MySQL clients
- TiKV provides distributed storage
- PD manages cluster metadata
- Horizontal scaling supported

## Troubleshooting

- TiKV store not available error: Verify PD service is running and accessible on port 2379, check network connectivity between containers
- Region unavailable during queries: Ensure at least 3 TiKV nodes are running for proper Raft majority, check TiKV logs for disk space issues
- Connection refused on port 4000: Wait for TiDB to complete initialization after TiKV cluster is ready, check TiDB logs for PD connection status
- High query latency in Grafana dashboards: Monitor TiKV disk I/O metrics, consider adding more TiKV nodes or optimizing query patterns
- PD leader election failures: Verify system clock synchronization across containers, ensure PD data directory has proper write permissions
- Prometheus metrics missing for TiDB components: Check that components are exposing metrics on their status ports (10080 for TiDB, 20180 for TiKV)

