docker.recipes

Temporal Workflow Engine

advanced

Temporal durable execution platform for microservices orchestration.

Overview

Temporal is a durable execution platform that guarantees workflow completion even in the face of failures, timeouts, and server crashes. Originally developed at Uber as Cadence, Temporal provides a programming model where developers write workflows as simple code that can run for months or years, automatically handling retries, timeouts, and state persistence. This makes it invaluable for complex business processes like payment processing, order fulfillment, and data pipeline orchestration. This stack combines Temporal's core server with PostgreSQL for durable state storage and Elasticsearch for advanced workflow visibility and search capabilities. PostgreSQL ensures ACID compliance for workflow state persistence, while Elasticsearch enables powerful querying across workflow executions, custom search attributes, and advanced filtering. The Temporal UI provides real-time visibility into running workflows, execution history, and failure analysis. This configuration suits teams building reliable distributed systems who need guarantees around long-running business processes. Unlike traditional message queues or basic orchestrators, Temporal handles complex scenarios like human-in-the-loop workflows, saga patterns, and multi-step transactions that span weeks or months. The PostgreSQL backend provides enterprise-grade reliability, while Elasticsearch transforms workflow monitoring from basic logging into sophisticated business intelligence.

Key Features

  • Durable execution with automatic retry logic and exponential backoff strategies
  • Advanced workflow visibility through Elasticsearch-powered search and filtering
  • PostgreSQL-backed state persistence ensuring ACID compliance for workflow data
  • Multi-language SDK support with consistent APIs across Go, Java, Python, and TypeScript
  • Real-time workflow monitoring via Temporal UI with execution timeline visualization
  • Custom search attributes stored in Elasticsearch for business-specific workflow queries
  • Automatic database schema setup and migration through temporalio/auto-setup image
  • gRPC API on port 7233 enabling high-performance client connections

Common Use Cases

  • 1E-commerce order processing with payment authorization, inventory checks, and fulfillment coordination
  • 2Financial transaction processing requiring multi-step approval workflows and regulatory compliance
  • 3Data pipeline orchestration for ETL processes with dependency management and failure recovery
  • 4Human-in-the-loop workflows for content moderation, approval processes, and manual interventions
  • 5Microservices saga patterns ensuring distributed transaction consistency across service boundaries
  • 6Scheduled job processing with complex timing requirements and cross-system coordination
  • 7Infrastructure automation workflows for deployment pipelines and environment provisioning

Prerequisites

  • Minimum 4GB RAM (2GB for Elasticsearch, 1GB for PostgreSQL, 1GB for Temporal server)
  • Docker Engine 20.10+ with docker-compose v2 support for health check dependencies
  • Available ports 7233 (Temporal gRPC), 8080 (UI), 5432 (PostgreSQL), 9200 (Elasticsearch)
  • Understanding of workflow concepts like activities, signals, queries, and temporal failure handling
  • Basic knowledge of your preferred Temporal SDK language (Go, Java, Python, or TypeScript)
  • POSTGRES_PASSWORD environment variable configured for database authentication

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  postgres: 
3    image: postgres:16-alpine
4    container_name: temporal-db
5    environment: 
6      - POSTGRES_USER=temporal
7      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
8    volumes: 
9      - postgres_data:/var/lib/postgresql/data
10    healthcheck: 
11      test: ["CMD-SHELL", "pg_isready -U temporal"]
12      interval: 10s
13      timeout: 5s
14      retries: 5
15    networks: 
16      - temporal-network
17
18  elasticsearch: 
19    image: elasticsearch:7.17.16
20    container_name: temporal-es
21    environment: 
22      - discovery.type=single-node
23      - xpack.security.enabled=false
24      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
25    volumes: 
26      - es_data:/usr/share/elasticsearch/data
27    networks: 
28      - temporal-network
29
30  temporal: 
31    image: temporalio/auto-setup:1.22
32    container_name: temporal
33    environment: 
34      - DB=postgres12
35      - DB_PORT=5432
36      - POSTGRES_USER=temporal
37      - POSTGRES_PWD=${POSTGRES_PASSWORD}
38      - POSTGRES_SEEDS=postgres
39      - ENABLE_ES=true
40      - ES_SEEDS=elasticsearch
41      - ES_VERSION=v7
42    ports: 
43      - "7233:7233"
44    depends_on: 
45      postgres: 
46        condition: service_healthy
47      elasticsearch: 
48        condition: service_started
49    networks: 
50      - temporal-network
51
52  temporal-ui: 
53    image: temporalio/ui:2.22.0
54    container_name: temporal-ui
55    environment: 
56      - TEMPORAL_ADDRESS=temporal:7233
57      - TEMPORAL_CORS_ORIGINS=http://localhost:3000
58    ports: 
59      - "8080:8080"
60    depends_on: 
61      - temporal
62    networks: 
63      - temporal-network
64
65  temporal-admin-tools: 
66    image: temporalio/admin-tools:1.22
67    container_name: temporal-admin
68    environment: 
69      - TEMPORAL_ADDRESS=temporal:7233
70    stdin_open: true
71    tty: true
72    depends_on: 
73      - temporal
74    networks: 
75      - temporal-network
76
77volumes: 
78  postgres_data: 
79  es_data: 
80
81networks: 
82  temporal-network: 
83    driver: bridge

.env Template

.env
1# Temporal Workflow Engine
2POSTGRES_PASSWORD=temporal_password

Usage Notes

  1. 1Temporal UI at http://localhost:8080
  2. 2gRPC API at localhost:7233
  3. 3Use admin-tools container for CLI
  4. 4Supports Go, Java, Python, TypeScript SDKs
  5. 5Elasticsearch enables advanced visibility

Individual Services(5 services)

Copy individual services to mix and match with your existing compose files.

postgres
postgres:
  image: postgres:16-alpine
  container_name: temporal-db
  environment:
    - POSTGRES_USER=temporal
    - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
  volumes:
    - postgres_data:/var/lib/postgresql/data
  healthcheck:
    test:
      - CMD-SHELL
      - pg_isready -U temporal
    interval: 10s
    timeout: 5s
    retries: 5
  networks:
    - temporal-network
elasticsearch
elasticsearch:
  image: elasticsearch:7.17.16
  container_name: temporal-es
  environment:
    - discovery.type=single-node
    - xpack.security.enabled=false
    - ES_JAVA_OPTS=-Xms512m -Xmx512m
  volumes:
    - es_data:/usr/share/elasticsearch/data
  networks:
    - temporal-network
temporal
temporal:
  image: temporalio/auto-setup:1.22
  container_name: temporal
  environment:
    - DB=postgres12
    - DB_PORT=5432
    - POSTGRES_USER=temporal
    - POSTGRES_PWD=${POSTGRES_PASSWORD}
    - POSTGRES_SEEDS=postgres
    - ENABLE_ES=true
    - ES_SEEDS=elasticsearch
    - ES_VERSION=v7
  ports:
    - "7233:7233"
  depends_on:
    postgres:
      condition: service_healthy
    elasticsearch:
      condition: service_started
  networks:
    - temporal-network
temporal-ui
temporal-ui:
  image: temporalio/ui:2.22.0
  container_name: temporal-ui
  environment:
    - TEMPORAL_ADDRESS=temporal:7233
    - TEMPORAL_CORS_ORIGINS=http://localhost:3000
  ports:
    - "8080:8080"
  depends_on:
    - temporal
  networks:
    - temporal-network
temporal-admin-tools
temporal-admin-tools:
  image: temporalio/admin-tools:1.22
  container_name: temporal-admin
  environment:
    - TEMPORAL_ADDRESS=temporal:7233
  stdin_open: true
  tty: true
  depends_on:
    - temporal
  networks:
    - temporal-network

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  postgres:
5    image: postgres:16-alpine
6    container_name: temporal-db
7    environment:
8      - POSTGRES_USER=temporal
9      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
10    volumes:
11      - postgres_data:/var/lib/postgresql/data
12    healthcheck:
13      test: ["CMD-SHELL", "pg_isready -U temporal"]
14      interval: 10s
15      timeout: 5s
16      retries: 5
17    networks:
18      - temporal-network
19
20  elasticsearch:
21    image: elasticsearch:7.17.16
22    container_name: temporal-es
23    environment:
24      - discovery.type=single-node
25      - xpack.security.enabled=false
26      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
27    volumes:
28      - es_data:/usr/share/elasticsearch/data
29    networks:
30      - temporal-network
31
32  temporal:
33    image: temporalio/auto-setup:1.22
34    container_name: temporal
35    environment:
36      - DB=postgres12
37      - DB_PORT=5432
38      - POSTGRES_USER=temporal
39      - POSTGRES_PWD=${POSTGRES_PASSWORD}
40      - POSTGRES_SEEDS=postgres
41      - ENABLE_ES=true
42      - ES_SEEDS=elasticsearch
43      - ES_VERSION=v7
44    ports:
45      - "7233:7233"
46    depends_on:
47      postgres:
48        condition: service_healthy
49      elasticsearch:
50        condition: service_started
51    networks:
52      - temporal-network
53
54  temporal-ui:
55    image: temporalio/ui:2.22.0
56    container_name: temporal-ui
57    environment:
58      - TEMPORAL_ADDRESS=temporal:7233
59      - TEMPORAL_CORS_ORIGINS=http://localhost:3000
60    ports:
61      - "8080:8080"
62    depends_on:
63      - temporal
64    networks:
65      - temporal-network
66
67  temporal-admin-tools:
68    image: temporalio/admin-tools:1.22
69    container_name: temporal-admin
70    environment:
71      - TEMPORAL_ADDRESS=temporal:7233
72    stdin_open: true
73    tty: true
74    depends_on:
75      - temporal
76    networks:
77      - temporal-network
78
79volumes:
80  postgres_data:
81  es_data:
82
83networks:
84  temporal-network:
85    driver: bridge
86EOF
87
88# 2. Create the .env file
89cat > .env << 'EOF'
90# Temporal Workflow Engine
91POSTGRES_PASSWORD=temporal_password
92EOF
93
94# 3. Start the services
95docker compose up -d
96
97# 4. View logs
98docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/temporal-workflow/run | bash

Troubleshooting

  • Temporal server fails to connect to PostgreSQL: Ensure POSTGRES_PASSWORD environment variable matches between postgres and temporal services
  • Elasticsearch container exits with 'max virtual memory areas vm.max_map_count too low': Run 'sudo sysctl -w vm.max_map_count=262144' on Docker host
  • Temporal UI shows 'Unable to connect to server': Verify temporal container is healthy and gRPC port 7233 is accessible within temporal-network
  • Workflows not appearing in search results: Check Elasticsearch health at temporal:9200/_cluster/health and verify ES_SEEDS points to elasticsearch service
  • Admin tools commands fail with permission errors: Use 'docker exec -it temporal-admin tctl' to run Temporal CLI commands within the container context
  • Database connection pool exhaustion under load: Increase PostgreSQL max_connections and configure connection pooling in POSTGRES_SEEDS environment

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

temporaltemporal-uitemporal-adminpostgreselasticsearch

Tags

#temporal#workflow#orchestration#microservices#durable-execution

Category

DevOps & CI/CD

Related Recipes

Ad Space