# Varnish HTTP Cache Stack

> High-performance HTTP cache with Varnish and monitoring.

- Category: Web Servers & Reverse Proxies
- Difficulty: intermediate
- Components: varnish, nginx, prometheus, grafana
- Tags: varnish, cache, http, cdn, performance
- Source: https://docker.recipes/web-servers/varnish-cache-complete

## Overview

Varnish is a high-performance HTTP accelerator and reverse proxy cache originally developed at the Norwegian newspaper Verdens Gang to handle their massive web traffic. Unlike traditional caches that store data on disk, Varnish stores cached content directly in RAM, delivering microsecond response times and handling thousands of requests per second. Its VCL (Varnish Configuration Language) provides granular control over caching policies, allowing developers to define complex rules for what gets cached, for how long, and under what conditions.

This stack combines Varnish's caching prowess with NGINX as the backend web server, creating a two-tier architecture where Varnish handles cache hits at lightning speed while NGINX serves the origin content. Prometheus collects detailed metrics from both Varnish and the custom varnish-exporter, tracking cache hit ratios, backend response times, and memory usage patterns. Grafana transforms this telemetry into visual dashboards that reveal cache performance bottlenecks and optimization opportunities.

Content delivery networks, high-traffic publishing platforms, and e-commerce sites benefit most from this configuration. The stack excels when serving content with predictable caching patterns - news articles, product catalogs, API responses, and static assets. With Varnish's ability to serve cached content even when backends fail, combined with comprehensive monitoring, this setup provides both performance gains and operational visibility that traditional web servers alone cannot match.

## Features

- VCL-based cache policies with custom purging rules and conditional caching logic
- RAM-based storage delivering sub-millisecond response times for cached content
- Grace mode serves stale content when NGINX backend becomes unavailable
- Prometheus varnish-exporter provides cache hit ratios and backend health metrics
- NGINX backend handles dynamic content generation and SSL termination
- Grafana dashboards visualize cache efficiency and identify optimization opportunities
- ESI (Edge Side Includes) support for dynamic content assembly at the cache layer
- Built-in load balancing across multiple NGINX backends with health checks

## Use cases

- High-traffic news websites caching article pages and reducing database load
- E-commerce platforms accelerating product catalog and search result delivery
- API gateways caching frequently requested endpoints and reducing backend calls
- WordPress and CMS hosting providers improving site performance for multiple clients
- CDN edge nodes serving regional content with detailed performance monitoring
- Development environments testing cache invalidation strategies before production
- Content aggregation platforms serving millions of article views with minimal infrastructure

## Prerequisites

- Minimum 1GB RAM (Varnish cache: 256MB + Prometheus: 256MB + Grafana: 512MB)
- Ports 80, 3000, 9090, and 9131 available on the host system
- Understanding of HTTP caching headers (Cache-Control, ETag, Vary)
- Basic VCL syntax knowledge for customizing Varnish cache behavior
- Familiarity with Prometheus query language (PromQL) for custom metrics
- Docker Compose 3.8+ with volume and network support

## docker-compose.yml

```yaml
services:
  nginx:
    image: nginx:alpine
    volumes:
      - ./html:/usr/share/nginx/html:ro
      - ./nginx.conf:/etc/nginx/nginx.conf:ro
    networks:
      - varnish_net

  varnish:
    image: varnish:stable
    ports:
      - "80:80"
    environment:
      - VARNISH_SIZE=256M
    volumes:
      - ./default.vcl:/etc/varnish/default.vcl:ro
    command: -a :80 -f /etc/varnish/default.vcl -s malloc,256M
    depends_on:
      - nginx
    networks:
      - varnish_net

  varnish-exporter:
    image: jonnenauha/prometheus_varnish_exporter:latest
    ports:
      - "9131:9131"
    command:
      - -varnish.health-url=http://varnish/health
    depends_on:
      - varnish
    networks:
      - varnish_net

  prometheus:
    image: prom/prometheus:latest
    ports:
      - "9090:9090"
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
      - prometheus_data:/prometheus
    networks:
      - varnish_net

  grafana:
    image: grafana/grafana:latest
    ports:
      - "3000:3000"
    environment:
      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
    volumes:
      - grafana_data:/var/lib/grafana
    networks:
      - varnish_net

volumes:
  prometheus_data:
  grafana_data:

networks:
  varnish_net:
```

## .env

```bash
# Varnish Cache
GRAFANA_PASSWORD=secure_grafana_password

# Varnish at http://localhost
# Grafana at http://localhost:3000
```

## Usage notes

- Varnish at http://localhost
- Configure VCL for cache rules
- Monitor with Grafana
- Varnish stats via varnishstat
- Great for static content caching

## Troubleshooting

- Varnish returns 503 Backend fetch failed: Check NGINX container health and verify network connectivity between varnish and nginx services
- Cache hit ratio remains low despite configuration: Review VCL rules in default.vcl and ensure backend responses include proper Cache-Control headers
- Grafana dashboard shows no Varnish metrics: Verify varnish-exporter container is running and accessible at port 9131, check prometheus.yml scrape configuration
- Memory usage grows continuously in Varnish: Reduce VARNISH_SIZE environment variable or implement cache object TTL limits in VCL configuration
- Prometheus target appears down for varnish-exporter: Update varnish-exporter health URL to match actual Varnish service name and port in Docker network
- NGINX backend connection refused: Ensure nginx service starts before Varnish and verify nginx.conf backend server configuration matches service names

