# Zipkin Tracing

> Distributed tracing system for latency analysis in microservices.

- Category: Monitoring & Observability
- Difficulty: beginner
- Components: zipkin, elasticsearch
- Tags: tracing, zipkin, observability, latency, microservices
- Source: https://docker.recipes/monitoring/zipkin-tracing

## Overview

Zipkin is an open-source distributed tracing system originally developed by Twitter to help troubleshoot latency problems in microservice architectures. It collects timing data needed to troubleshoot latency issues, allowing developers to visualize request flows across multiple services and identify performance bottlenecks. Zipkin follows the OpenTracing standard and provides detailed insights into how requests propagate through distributed systems, making it essential for maintaining observable microservices. This configuration combines Zipkin with Elasticsearch as the storage backend, creating a powerful tracing platform that can handle high-volume trace data with advanced search and analytics capabilities. Elasticsearch provides Zipkin with scalable storage, fast querying, and the ability to perform complex aggregations on trace data, while the zipkin-dependencies service automatically generates service dependency graphs from collected traces. Development teams building microservices, DevOps engineers managing distributed systems, and organizations transitioning from monolithic to service-oriented architectures will find this stack invaluable for understanding system behavior and optimizing performance across service boundaries.

## Features

- B3 propagation support for seamless trace context propagation across HTTP requests
- Real-time service dependency graph generation with automatic topology discovery
- Latency percentile analysis with P50, P95, and P99 calculations for performance monitoring
- Elasticsearch-powered full-text search across trace annotations and tags
- Automatic trace retention and cleanup through Elasticsearch index lifecycle management
- Support for multiple span types including client, server, producer, and consumer spans
- Built-in sampling strategies to control trace collection overhead in high-traffic environments
- RESTful API for programmatic trace submission and querying

## Use cases

- Debugging performance bottlenecks in microservice architectures during development
- Root cause analysis for intermittent latency spikes in production environments
- Service dependency mapping for legacy system modernization projects
- Performance regression testing in CI/CD pipelines with trace comparison
- Capacity planning by analyzing request patterns and service utilization
- SLA monitoring and alerting based on service-level latency metrics
- API gateway performance optimization through upstream service analysis

## Prerequisites

- Minimum 2GB RAM for Elasticsearch storage backend (4GB recommended for production)
- Port 9411 available for Zipkin web UI and API access
- Basic understanding of distributed tracing concepts and span terminology
- Application instrumentation with Zipkin-compatible tracing libraries
- Docker Engine 19.03+ with Docker Compose v2 support
- At least 10GB free disk space for trace data storage and Elasticsearch indices

## docker-compose.yml

```yaml
services:
  zipkin:
    image: openzipkin/zipkin:latest
    container_name: zipkin
    environment:
      - STORAGE_TYPE=elasticsearch
      - ES_HOSTS=elasticsearch:9200
    ports:
      - "9411:9411"
    depends_on:
      - elasticsearch
    networks:
      - zipkin-network
    restart: unless-stopped

  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
    container_name: zipkin-elasticsearch
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    volumes:
      - elasticsearch-data:/usr/share/elasticsearch/data
    networks:
      - zipkin-network
    restart: unless-stopped

  zipkin-dependencies:
    image: openzipkin/zipkin-dependencies:latest
    container_name: zipkin-dependencies
    environment:
      - STORAGE_TYPE=elasticsearch
      - ES_HOSTS=elasticsearch:9200
    depends_on:
      - elasticsearch
    networks:
      - zipkin-network
    entrypoint: crond -f

volumes:
  elasticsearch-data:

networks:
  zipkin-network:
    driver: bridge
```

## .env

```bash
# Zipkin
# Send traces to http://localhost:9411/api/v2/spans
```

## Usage notes

- Zipkin UI at http://localhost:9411
- Send traces via HTTP POST
- Service dependency graph
- Latency percentile analysis
- Supports B3 propagation

## Troubleshooting

- Zipkin UI shows no traces: Verify applications are instrumented and sending traces to http://localhost:9411/api/v2/spans
- Elasticsearch connection refused errors: Wait for Elasticsearch to fully start (30-60 seconds) before Zipkin attempts connection
- Out of memory errors in Elasticsearch: Increase ES_JAVA_OPTS heap size from 512m to 1g or higher based on trace volume
- zipkin-dependencies service not generating dependency graphs: Check that spans have proper service names and that Elasticsearch contains trace data
- High CPU usage on zipkin-dependencies: Adjust cron schedule in entrypoint or limit dependency analysis to specific time ranges
- Traces not appearing in UI after submission: Check Elasticsearch cluster health and verify STORAGE_TYPE environment variable is set correctly

