docker.recipes

QuestDB Real-time Analytics

intermediate

QuestDB time-series database with Grafana dashboards.

Overview

QuestDB is a high-performance, open-source time-series database designed for real-time analytics and IoT applications. Built from the ground up in Java and C++, QuestDB leverages columnar storage and vectorized execution to deliver exceptional ingestion rates (over 4 million rows per second) and sub-millisecond query performance. It supports both SQL queries and the InfluxDB Line Protocol for data ingestion, making it compatible with existing monitoring tools and applications. This stack combines QuestDB's powerful time-series capabilities with Grafana's visualization platform and NGINX as a reverse proxy. QuestDB handles high-frequency data ingestion through its ILP port while serving analytical queries via its PostgreSQL wire protocol, Grafana connects directly to QuestDB to create real-time dashboards and alerting, and NGINX provides load balancing and SSL termination for production deployments. This architecture enables organizations to collect, store, and visualize massive amounts of time-series data with minimal latency. DevOps engineers building monitoring infrastructure, IoT developers processing sensor data, and financial institutions tracking market data will find this combination particularly valuable. QuestDB's unique approach to time-series storage, combined with Grafana's extensive visualization options and NGINX's proven reliability, creates a robust analytics platform that scales from startup prototypes to enterprise-grade deployments handling billions of data points daily.

Key Features

  • QuestDB's columnar storage engine with SIMD-optimized queries for sub-millisecond response times
  • Native InfluxDB Line Protocol support on port 9009 for high-frequency data ingestion
  • PostgreSQL wire protocol compatibility enabling direct Grafana data source connections
  • QuestDB's built-in web console for interactive SQL queries and database management
  • Grafana's time-series specific visualizations including time series panels and stat panels
  • NGINX reverse proxy configuration for load balancing across QuestDB instances
  • QuestDB's automatic data partitioning by time intervals for optimized storage and queries
  • Grafana alerting integration with QuestDB's real-time query capabilities

Common Use Cases

  • 1IoT sensor data collection and real-time monitoring dashboards for manufacturing facilities
  • 2Financial market data ingestion and trading analytics with millisecond-latency requirements
  • 3Application performance monitoring with high-cardinality metrics and custom Grafana panels
  • 4Infrastructure monitoring collecting system metrics from thousands of servers
  • 5Real-time business intelligence dashboards for e-commerce transaction analysis
  • 6Cryptocurrency trading platforms requiring high-frequency price data storage and visualization
  • 7Smart city initiatives processing traffic, environmental, and utility sensor data

Prerequisites

  • Docker Engine 20.10+ and Docker Compose 2.0+ for container orchestration
  • Minimum 2GB RAM available (QuestDB: 512MB, Grafana: 512MB, NGINX: 256MB)
  • Ports 9000, 8812, 9009, 3000, and 80 available on the host system
  • Basic understanding of SQL queries and time-series data concepts
  • Familiarity with Grafana dashboard creation and data source configuration
  • Understanding of InfluxDB Line Protocol format for data ingestion

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  questdb: 
3    image: questdb/questdb:latest
4    container_name: questdb
5    restart: unless-stopped
6    ports: 
7      - "${QDB_HTTP_PORT:-9000}:9000"
8      - "${QDB_PG_PORT:-8812}:8812"
9      - "${QDB_ILP_PORT:-9009}:9009"
10    volumes: 
11      - questdb_data:/var/lib/questdb
12
13  grafana: 
14    image: grafana/grafana:latest
15    container_name: qdb-grafana
16    restart: unless-stopped
17    ports: 
18      - "${GRAFANA_PORT:-3000}:3000"
19    environment: 
20      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
21    volumes: 
22      - grafana_data:/var/lib/grafana
23
24  nginx: 
25    image: nginx:alpine
26    container_name: qdb-nginx
27    restart: unless-stopped
28    ports: 
29      - "${NGINX_PORT:-80}:80"
30    volumes: 
31      - ./nginx.conf:/etc/nginx/nginx.conf:ro
32
33volumes: 
34  questdb_data: 
35  grafana_data:

.env Template

.env
1# QuestDB Analytics
2QDB_HTTP_PORT=9000
3QDB_PG_PORT=8812
4QDB_ILP_PORT=9009
5GRAFANA_PORT=3000
6GRAFANA_PASSWORD=admin
7NGINX_PORT=80

Usage Notes

  1. 1QuestDB Console at http://localhost:9000
  2. 2PostgreSQL wire on port 8812
  3. 3ILP for high-frequency inserts on 9009
  4. 4Grafana at http://localhost:3000

Individual Services(3 services)

Copy individual services to mix and match with your existing compose files.

questdb
questdb:
  image: questdb/questdb:latest
  container_name: questdb
  restart: unless-stopped
  ports:
    - ${QDB_HTTP_PORT:-9000}:9000
    - ${QDB_PG_PORT:-8812}:8812
    - ${QDB_ILP_PORT:-9009}:9009
  volumes:
    - questdb_data:/var/lib/questdb
grafana
grafana:
  image: grafana/grafana:latest
  container_name: qdb-grafana
  restart: unless-stopped
  ports:
    - ${GRAFANA_PORT:-3000}:3000
  environment:
    - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
  volumes:
    - grafana_data:/var/lib/grafana
nginx
nginx:
  image: nginx:alpine
  container_name: qdb-nginx
  restart: unless-stopped
  ports:
    - ${NGINX_PORT:-80}:80
  volumes:
    - ./nginx.conf:/etc/nginx/nginx.conf:ro

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  questdb:
5    image: questdb/questdb:latest
6    container_name: questdb
7    restart: unless-stopped
8    ports:
9      - "${QDB_HTTP_PORT:-9000}:9000"
10      - "${QDB_PG_PORT:-8812}:8812"
11      - "${QDB_ILP_PORT:-9009}:9009"
12    volumes:
13      - questdb_data:/var/lib/questdb
14
15  grafana:
16    image: grafana/grafana:latest
17    container_name: qdb-grafana
18    restart: unless-stopped
19    ports:
20      - "${GRAFANA_PORT:-3000}:3000"
21    environment:
22      - GF_SECURITY_ADMIN_PASSWORD=${GRAFANA_PASSWORD}
23    volumes:
24      - grafana_data:/var/lib/grafana
25
26  nginx:
27    image: nginx:alpine
28    container_name: qdb-nginx
29    restart: unless-stopped
30    ports:
31      - "${NGINX_PORT:-80}:80"
32    volumes:
33      - ./nginx.conf:/etc/nginx/nginx.conf:ro
34
35volumes:
36  questdb_data:
37  grafana_data:
38EOF
39
40# 2. Create the .env file
41cat > .env << 'EOF'
42# QuestDB Analytics
43QDB_HTTP_PORT=9000
44QDB_PG_PORT=8812
45QDB_ILP_PORT=9009
46GRAFANA_PORT=3000
47GRAFANA_PASSWORD=admin
48NGINX_PORT=80
49EOF
50
51# 3. Start the services
52docker compose up -d
53
54# 4. View logs
55docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/questdb-realtime-stack/run | bash

Troubleshooting

  • QuestDB 'cairo.exception.CairoException: could not open read-only': Ensure questdb_data volume has correct permissions (chown 10001:10001)
  • Grafana 'Bad Gateway' when connecting to QuestDB: Verify QuestDB PostgreSQL port 8812 is accessible and use 'questdb:8812' as host in data source
  • High memory usage during QuestDB ingestion: Increase commit lag settings in QuestDB configuration or reduce batch sizes in ILP client
  • QuestDB queries timing out in Grafana: Add query timeout parameters and consider adding time range filters to dashboard queries
  • NGINX 502 errors: Check that upstream QuestDB and Grafana containers are running and healthy using docker ps
  • QuestDB ILP ingestion failures: Verify client is sending properly formatted InfluxDB Line Protocol and check QuestDB logs for parsing errors

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

questdbgrafananginx

Tags

#questdb#time-series#analytics#real-time

Category

Database Stacks

Related Recipes

Ad Space