Kestra

Key Features

• Event-driven workflow triggers supporting schedules, webhooks, file watchers, Kafka consumers, and database change streams
• Visual workflow topology with real-time execution monitoring and interactive dependency graphs
• Built-in secrets management with variable templating and environment-specific configuration inheritance
• PostgreSQL-backed execution history with queryable logs, metrics, and audit trails for compliance reporting
• Plugin ecosystem with 400+ pre-built connectors for databases, cloud platforms, APIs, and data processing tools
• Multi-tenant namespace isolation with role-based access control and team-based workflow organization
• Advanced workflow patterns including parallel execution, conditional branching, error handling, and retry policies
• RESTful API on port 8081 enabling programmatic workflow management and integration with external systems

Common Use Cases

• 1Data pipeline orchestration replacing Airflow with event-driven ETL processes triggered by file uploads or database changes
• 2Microservice workflow coordination responding to Kafka events for order processing, user onboarding, or payment workflows
• 3Infrastructure automation triggered by monitoring alerts, combining Terraform deployments with notification workflows
• 4CI/CD pipeline orchestration beyond traditional build tools, handling complex deployment workflows across multiple environments
• 5Business process automation connecting CRM webhooks to email campaigns, invoice generation, and customer notification sequences
• 6IoT data processing workflows triggered by sensor data streams, combining real-time analytics with automated responses
• 7Multi-cloud resource management coordinating deployments, backups, and cost optimization across AWS, GCP, and Azure

Prerequisites

• Docker Engine 20.10+ with Docker Compose v2 for container orchestration support
• Minimum 2GB RAM allocated to Docker (1GB+ for PostgreSQL, additional overhead for Kestra workflows)
• Available ports 8080 (Kestra Web UI) and 8081 (Kestra API) on the Docker host
• Environment variables DB_USER and DB_PASSWORD configured for PostgreSQL authentication
• Basic understanding of YAML syntax for workflow definition and event-driven architecture concepts
• Familiarity with PostgreSQL administration for backup strategies and performance tuning in production environments

docker-compose.yml

1services: 
2  kestra: 
3    image: kestra/kestra:latest
4    container_name: kestra
5    restart: unless-stopped
6    command: server standalone
7    environment: 
8      KESTRA_CONFIGURATION: |
9        datasources: 
10          postgres: 
11            url: jdbc:postgresql://postgres:5432/kestra
12            driverClassName: org.postgresql.Driver
13            username: ${DB_USER}
14            password: ${DB_PASSWORD}
15    ports: 
16      - "8080:8080"
17      - "8081:8081"
18    depends_on: 
19      - postgres
20    networks: 
21      - kestra
22
23  postgres: 
24    image: postgres:16-alpine
25    container_name: kestra-postgres
26    environment: 
27      POSTGRES_DB: kestra
28      POSTGRES_USER: ${DB_USER}
29      POSTGRES_PASSWORD: ${DB_PASSWORD}
30    volumes: 
31      - postgres_data:/var/lib/postgresql/data
32    networks: 
33      - kestra
34
35volumes: 
36  postgres_data: 
37
38networks: 
39  kestra: 
40    driver: bridge

.env Template

1DB_USER=kestra
2DB_PASSWORD=changeme

Usage Notes

Individual Services(2 services)

Copy individual services to mix and match with your existing compose files.

kestra:
  image: kestra/kestra:latest
  container_name: kestra
  restart: unless-stopped
  command: server standalone
  environment:
    KESTRA_CONFIGURATION: |
      datasources:
        postgres:
          url: jdbc:postgresql://postgres:5432/kestra
          driverClassName: org.postgresql.Driver
          username: ${DB_USER}
          password: ${DB_PASSWORD}
  ports:
    - "8080:8080"
    - "8081:8081"
  depends_on:
    - postgres
  networks:
    - kestra

postgres:
  image: postgres:16-alpine
  container_name: kestra-postgres
  environment:
    POSTGRES_DB: kestra
    POSTGRES_USER: ${DB_USER}
    POSTGRES_PASSWORD: ${DB_PASSWORD}
  volumes:
    - postgres_data:/var/lib/postgresql/data
  networks:
    - kestra

Quick Start

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  kestra:
5    image: kestra/kestra:latest
6    container_name: kestra
7    restart: unless-stopped
8    command: server standalone
9    environment:
10      KESTRA_CONFIGURATION: |
11        datasources:
12          postgres:
13            url: jdbc:postgresql://postgres:5432/kestra
14            driverClassName: org.postgresql.Driver
15            username: ${DB_USER}
16            password: ${DB_PASSWORD}
17    ports:
18      - "8080:8080"
19      - "8081:8081"
20    depends_on:
21      - postgres
22    networks:
23      - kestra
24
25  postgres:
26    image: postgres:16-alpine
27    container_name: kestra-postgres
28    environment:
29      POSTGRES_DB: kestra
30      POSTGRES_USER: ${DB_USER}
31      POSTGRES_PASSWORD: ${DB_PASSWORD}
32    volumes:
33      - postgres_data:/var/lib/postgresql/data
34    networks:
35      - kestra
36
37volumes:
38  postgres_data:
39
40networks:
41  kestra:
42    driver: bridge
43EOF
44
45# 2. Create the .env file
46cat > .env << 'EOF'
47DB_USER=kestra
48DB_PASSWORD=changeme
49EOF
50
51# 3. Start the services
52docker compose up -d
53
54# 4. View logs
55docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

1curl -fsSL https://docker.recipes/api/recipes/kestra/run | bash

Troubleshooting