# Temporal Workflow Engine

> Temporal durable execution platform for microservices orchestration.

- Category: DevOps & CI/CD
- Difficulty: advanced
- Components: temporal, temporal-ui, temporal-admin, postgres, elasticsearch
- Tags: temporal, workflow, orchestration, microservices, durable-execution
- Source: https://docker.recipes/devops/temporal-workflow

## Overview

Temporal is a durable execution platform that guarantees workflow completion even in the face of failures, timeouts, and server crashes. Originally developed at Uber as Cadence, Temporal provides a programming model where developers write workflows as simple code that can run for months or years, automatically handling retries, timeouts, and state persistence. This makes it invaluable for complex business processes like payment processing, order fulfillment, and data pipeline orchestration.

This stack combines Temporal's core server with PostgreSQL for durable state storage and Elasticsearch for advanced workflow visibility and search capabilities. PostgreSQL ensures ACID compliance for workflow state persistence, while Elasticsearch enables powerful querying across workflow executions, custom search attributes, and advanced filtering. The Temporal UI provides real-time visibility into running workflows, execution history, and failure analysis.

This configuration suits teams building reliable distributed systems who need guarantees around long-running business processes. Unlike traditional message queues or basic orchestrators, Temporal handles complex scenarios like human-in-the-loop workflows, saga patterns, and multi-step transactions that span weeks or months. The PostgreSQL backend provides enterprise-grade reliability, while Elasticsearch transforms workflow monitoring from basic logging into sophisticated business intelligence.

## Features

- Durable execution with automatic retry logic and exponential backoff strategies
- Advanced workflow visibility through Elasticsearch-powered search and filtering
- PostgreSQL-backed state persistence ensuring ACID compliance for workflow data
- Multi-language SDK support with consistent APIs across Go, Java, Python, and TypeScript
- Real-time workflow monitoring via Temporal UI with execution timeline visualization
- Custom search attributes stored in Elasticsearch for business-specific workflow queries
- Automatic database schema setup and migration through temporalio/auto-setup image
- gRPC API on port 7233 enabling high-performance client connections

## Use cases

- E-commerce order processing with payment authorization, inventory checks, and fulfillment coordination
- Financial transaction processing requiring multi-step approval workflows and regulatory compliance
- Data pipeline orchestration for ETL processes with dependency management and failure recovery
- Human-in-the-loop workflows for content moderation, approval processes, and manual interventions
- Microservices saga patterns ensuring distributed transaction consistency across service boundaries
- Scheduled job processing with complex timing requirements and cross-system coordination
- Infrastructure automation workflows for deployment pipelines and environment provisioning

## Prerequisites

- Minimum 4GB RAM (2GB for Elasticsearch, 1GB for PostgreSQL, 1GB for Temporal server)
- Docker Engine 20.10+ with docker-compose v2 support for health check dependencies
- Available ports 7233 (Temporal gRPC), 8080 (UI), 5432 (PostgreSQL), 9200 (Elasticsearch)
- Understanding of workflow concepts like activities, signals, queries, and temporal failure handling
- Basic knowledge of your preferred Temporal SDK language (Go, Java, Python, or TypeScript)
- POSTGRES_PASSWORD environment variable configured for database authentication

## docker-compose.yml

```yaml
services:
  postgres:
    image: postgres:16-alpine
    container_name: temporal-db
    environment:
      - POSTGRES_USER=temporal
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U temporal"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - temporal-network

  elasticsearch:
    image: elasticsearch:7.17.16
    container_name: temporal-es
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    volumes:
      - es_data:/usr/share/elasticsearch/data
    networks:
      - temporal-network

  temporal:
    image: temporalio/auto-setup:1.22
    container_name: temporal
    environment:
      - DB=postgres12
      - DB_PORT=5432
      - POSTGRES_USER=temporal
      - POSTGRES_PWD=${POSTGRES_PASSWORD}
      - POSTGRES_SEEDS=postgres
      - ENABLE_ES=true
      - ES_SEEDS=elasticsearch
      - ES_VERSION=v7
    ports:
      - "7233:7233"
    depends_on:
      postgres:
        condition: service_healthy
      elasticsearch:
        condition: service_started
    networks:
      - temporal-network

  temporal-ui:
    image: temporalio/ui:2.22.0
    container_name: temporal-ui
    environment:
      - TEMPORAL_ADDRESS=temporal:7233
      - TEMPORAL_CORS_ORIGINS=http://localhost:3000
    ports:
      - "8080:8080"
    depends_on:
      - temporal
    networks:
      - temporal-network

  temporal-admin-tools:
    image: temporalio/admin-tools:1.22
    container_name: temporal-admin
    environment:
      - TEMPORAL_ADDRESS=temporal:7233
    stdin_open: true
    tty: true
    depends_on:
      - temporal
    networks:
      - temporal-network

volumes:
  postgres_data:
  es_data:

networks:
  temporal-network:
    driver: bridge
```

## .env

```bash
# Temporal Workflow Engine
POSTGRES_PASSWORD=temporal_password
```

## Usage notes

- Temporal UI at http://localhost:8080
- gRPC API at localhost:7233
- Use admin-tools container for CLI
- Supports Go, Java, Python, TypeScript SDKs
- Elasticsearch enables advanced visibility

## Troubleshooting

- Temporal server fails to connect to PostgreSQL: Ensure POSTGRES_PASSWORD environment variable matches between postgres and temporal services
- Elasticsearch container exits with 'max virtual memory areas vm.max_map_count too low': Run 'sudo sysctl -w vm.max_map_count=262144' on Docker host
- Temporal UI shows 'Unable to connect to server': Verify temporal container is healthy and gRPC port 7233 is accessible within temporal-network
- Workflows not appearing in search results: Check Elasticsearch health at temporal:9200/_cluster/health and verify ES_SEEDS points to elasticsearch service
- Admin tools commands fail with permission errors: Use 'docker exec -it temporal-admin tctl' to run Temporal CLI commands within the container context
- Database connection pool exhaustion under load: Increase PostgreSQL max_connections and configure connection pooling in POSTGRES_SEEDS environment

