# HashiCorp Nomad Orchestration

> Workload orchestration with Nomad cluster, Consul integration, and UI.

- Category: DevOps & CI/CD
- Difficulty: advanced
- Components: nomad-server, nomad-client, consul, traefik
- Tags: nomad, hashicorp, orchestration, scheduling, jobs
- Source: https://docker.recipes/devops/nomad-cluster

## Overview

HashiCorp Nomad is a simple and flexible workload orchestrator that schedules containers, legacy applications, VMs, and batch jobs across on-premises and cloud environments. Unlike Kubernetes, Nomad focuses on simplicity with a single binary deployment while supporting multi-workload types beyond just containers. This makes it particularly attractive for organizations running mixed infrastructure or those seeking a lighter alternative to complex orchestration platforms.

This stack combines Nomad server and client nodes with Consul for service discovery and health checking, plus Traefik for automatic load balancing and routing. Consul acts as the backbone for service registration and configuration storage, while Traefik automatically discovers services registered in Consul and routes traffic accordingly. The configuration establishes a complete orchestration environment with one server node and two client nodes, providing job scheduling, service mesh capabilities, and dynamic traffic routing.

This setup is ideal for teams already invested in the HashiCorp ecosystem, organizations running heterogeneous workloads, or those transitioning from traditional infrastructure to container orchestration. The combination offers enterprise-grade scheduling with native integration between components, making it particularly valuable for companies requiring both container and non-container workload management in a unified platform.

## Features

- Multi-workload scheduling supporting Docker containers, VMs, Java applications, and batch jobs
- Native Consul integration for automatic service discovery and health checking
- Traefik auto-discovery of Nomad services with dynamic load balancing
- Single binary Nomad deployment with minimal operational overhead
- Job specification in HCL format with declarative infrastructure management
- Built-in federation capabilities for multi-datacenter deployments
- GPU workload support for machine learning and computational tasks
- Blue-green deployment strategies with rolling updates

## Use cases

- Mixed workload environments running containers alongside legacy applications
- Batch processing systems requiring scheduled computational jobs
- Organizations migrating from traditional infrastructure to orchestrated environments
- Development teams needing simpler orchestration than Kubernetes
- Multi-datacenter deployments requiring federated job scheduling
- GPU-accelerated workloads for machine learning and data processing
- HashiCorp stack users integrating with existing Vault and Terraform workflows

## Prerequisites

- Docker Engine with privileged container support enabled
- Minimum 2GB RAM available for server components (Nomad server requires 512MB, Consul 512MB)
- Ports 4646-4648, 8500, 80, and 8080 available on the host system
- Basic understanding of HCL configuration syntax for Nomad job specifications
- Knowledge of HashiCorp Consul concepts for service registration
- Familiarity with workload orchestration concepts and job scheduling

## docker-compose.yml

```yaml
services:
  consul:
    image: hashicorp/consul:latest
    command: agent -server -bootstrap-expect=1 -ui -client=0.0.0.0
    ports:
      - "8500:8500"
    volumes:
      - consul_data:/consul/data
    networks:
      - nomad_net

  nomad-server:
    image: hashicorp/nomad:latest
    command: agent -server -bootstrap-expect=1 -bind=0.0.0.0
    ports:
      - "4646:4646"
      - "4647:4647"
      - "4648:4648"
    volumes:
      - ./nomad/server.hcl:/etc/nomad.d/nomad.hcl
      - nomad_server:/opt/nomad/data
    privileged: true
    depends_on:
      - consul
    networks:
      - nomad_net

  nomad-client-1:
    image: hashicorp/nomad:latest
    command: agent -client -bind=0.0.0.0
    volumes:
      - ./nomad/client.hcl:/etc/nomad.d/nomad.hcl
      - nomad_client_1:/opt/nomad/data
      - /var/run/docker.sock:/var/run/docker.sock
    privileged: true
    depends_on:
      - nomad-server
      - consul
    networks:
      - nomad_net

  nomad-client-2:
    image: hashicorp/nomad:latest
    command: agent -client -bind=0.0.0.0
    volumes:
      - ./nomad/client.hcl:/etc/nomad.d/nomad.hcl
      - nomad_client_2:/opt/nomad/data
      - /var/run/docker.sock:/var/run/docker.sock
    privileged: true
    depends_on:
      - nomad-server
      - consul
    networks:
      - nomad_net

  traefik:
    image: traefik:v2.10
    command:
      - --api.insecure=true
      - --providers.consulcatalog=true
      - --providers.consulcatalog.endpoint.address=consul:8500
    ports:
      - "80:80"
      - "8080:8080"
    depends_on:
      - consul
    networks:
      - nomad_net

volumes:
  consul_data:
  nomad_server:
  nomad_client_1:
  nomad_client_2:

networks:
  nomad_net:
```

## .env

```bash
# Nomad Cluster
# Nomad UI at http://localhost:4646
# Consul UI at http://localhost:8500
# Traefik at http://localhost:8080
```

## Usage notes

- Nomad UI at http://localhost:4646
- Consul UI at http://localhost:8500
- Traefik for service routing
- Docker driver for containers
- Job specs in HCL format

## Troubleshooting

- Nomad client 'failed to fingerprint driver docker': Ensure Docker socket is properly mounted and accessible to Nomad containers
- Consul UI shows unhealthy services: Verify Nomad agents can reach Consul on port 8500 and check network connectivity
- Jobs stuck in pending state: Check client node resources and constraints, verify clients are connected to server
- Traefik not discovering services: Confirm Consul catalog contains service registrations and Traefik can access Consul endpoint
- Permission denied errors on job execution: Verify privileged mode is enabled for Nomad containers to access Docker socket
- Server bootstrap fails: Ensure only one server is starting initially and bootstrap-expect matches actual server count

