docker.recipes

Concourse CI Pipeline

intermediate

Concourse CI for container-native CI.

Overview

Concourse CI is a container-native continuous integration and continuous delivery (CI/CD) platform that treats pipelines as first-class citizens. Originally developed by Pivotal (now part of VMware Tanzu), Concourse emphasizes simplicity, reproducibility, and transparency in build processes. Unlike traditional CI systems that rely on persistent build agents, Concourse executes every task in ephemeral containers, ensuring clean, reproducible builds without accumulated state or configuration drift. This deployment creates a fully functional Concourse CI environment using two services: a PostgreSQL 15 database for metadata storage and the Concourse application server running in quickstart mode. The PostgreSQL instance stores pipeline configurations, build history, resource metadata, and team permissions, while the Concourse container handles both web UI and worker responsibilities in a single process. This configuration is ideal for development teams, small organizations, or proof-of-concept deployments that need a lightweight yet powerful CI/CD solution. The container-native approach makes this stack particularly valuable for teams already invested in containerized workflows, as pipelines can seamlessly integrate with existing Docker registries, Kubernetes clusters, and container-based deployment strategies.

Key Features

  • Container-native pipeline execution with ephemeral task containers ensuring clean, reproducible builds
  • Pipeline-as-code with YAML configuration stored in version control for full auditability
  • Resource-based pipeline modeling with built-in support for Git, Docker images, S3, and time triggers
  • Real-time pipeline visualization with interactive web interface showing resource flow and build status
  • Multi-team support with role-based access control and team-specific pipeline isolation
  • Built-in secret management with credential interpolation from various backends
  • Fly CLI integration for pipeline management, job triggering, and direct task execution
  • Automatic resource checking and version tracking across pipeline dependencies

Common Use Cases

  • 1Containerized application CI/CD with Docker image building, testing, and registry publishing
  • 2Microservices deployment pipelines with service-specific build stages and integration testing
  • 3Infrastructure-as-code validation and deployment using Terraform, Ansible, or Kubernetes manifests
  • 4Multi-environment promotion workflows with automated testing gates between development, staging, and production
  • 5Open source project automation including pull request validation, release packaging, and documentation generation
  • 6Development team onboarding and CI/CD learning environment with visual pipeline feedback
  • 7Compliance-focused deployments requiring full build traceability and approval workflows

Prerequisites

  • Docker Engine 20.10+ with privileged container support for Concourse worker functionality
  • Minimum 2GB RAM available (1GB for Concourse, 512MB for PostgreSQL, plus container overhead)
  • Port 8080 available for Concourse web interface access
  • Fly CLI installed locally for pipeline management and task execution
  • Basic understanding of YAML syntax for pipeline configuration
  • Familiarity with container registries and Git repositories for resource integration

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  concourse-db: 
3    image: postgres:15-alpine
4    container_name: concourse-db
5    restart: unless-stopped
6    environment: 
7      - POSTGRES_USER=concourse
8      - POSTGRES_PASSWORD=${DB_PASSWORD}
9      - POSTGRES_DB=concourse
10    volumes: 
11      - concourse_db_data:/var/lib/postgresql/data
12
13  concourse: 
14    image: concourse/concourse:latest
15    container_name: concourse
16    restart: unless-stopped
17    privileged: true
18    ports: 
19      - "${CONCOURSE_PORT:-8080}:8080"
20    environment: 
21      - CONCOURSE_POSTGRES_HOST=concourse-db
22      - CONCOURSE_POSTGRES_USER=concourse
23      - CONCOURSE_POSTGRES_PASSWORD=${DB_PASSWORD}
24      - CONCOURSE_EXTERNAL_URL=http://localhost:${CONCOURSE_PORT:-8080}
25      - CONCOURSE_ADD_LOCAL_USER=admin:${ADMIN_PASSWORD}
26      - CONCOURSE_MAIN_TEAM_LOCAL_USER=admin
27    command: quickstart
28    depends_on: 
29      - concourse-db
30
31volumes: 
32  concourse_db_data:

.env Template

.env
1# Concourse CI
2CONCOURSE_PORT=8080
3DB_PASSWORD=concourse_password
4ADMIN_PASSWORD=admin_password

Usage Notes

  1. 1Concourse at http://localhost:8080
  2. 2Login with admin credentials
  3. 3Install fly CLI for pipelines
  4. 4Define pipelines in YAML

Individual Services(2 services)

Copy individual services to mix and match with your existing compose files.

concourse-db
concourse-db:
  image: postgres:15-alpine
  container_name: concourse-db
  restart: unless-stopped
  environment:
    - POSTGRES_USER=concourse
    - POSTGRES_PASSWORD=${DB_PASSWORD}
    - POSTGRES_DB=concourse
  volumes:
    - concourse_db_data:/var/lib/postgresql/data
concourse
concourse:
  image: concourse/concourse:latest
  container_name: concourse
  restart: unless-stopped
  privileged: true
  ports:
    - ${CONCOURSE_PORT:-8080}:8080
  environment:
    - CONCOURSE_POSTGRES_HOST=concourse-db
    - CONCOURSE_POSTGRES_USER=concourse
    - CONCOURSE_POSTGRES_PASSWORD=${DB_PASSWORD}
    - CONCOURSE_EXTERNAL_URL=http://localhost:${CONCOURSE_PORT:-8080}
    - CONCOURSE_ADD_LOCAL_USER=admin:${ADMIN_PASSWORD}
    - CONCOURSE_MAIN_TEAM_LOCAL_USER=admin
  command: quickstart
  depends_on:
    - concourse-db

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  concourse-db:
5    image: postgres:15-alpine
6    container_name: concourse-db
7    restart: unless-stopped
8    environment:
9      - POSTGRES_USER=concourse
10      - POSTGRES_PASSWORD=${DB_PASSWORD}
11      - POSTGRES_DB=concourse
12    volumes:
13      - concourse_db_data:/var/lib/postgresql/data
14
15  concourse:
16    image: concourse/concourse:latest
17    container_name: concourse
18    restart: unless-stopped
19    privileged: true
20    ports:
21      - "${CONCOURSE_PORT:-8080}:8080"
22    environment:
23      - CONCOURSE_POSTGRES_HOST=concourse-db
24      - CONCOURSE_POSTGRES_USER=concourse
25      - CONCOURSE_POSTGRES_PASSWORD=${DB_PASSWORD}
26      - CONCOURSE_EXTERNAL_URL=http://localhost:${CONCOURSE_PORT:-8080}
27      - CONCOURSE_ADD_LOCAL_USER=admin:${ADMIN_PASSWORD}
28      - CONCOURSE_MAIN_TEAM_LOCAL_USER=admin
29    command: quickstart
30    depends_on:
31      - concourse-db
32
33volumes:
34  concourse_db_data:
35EOF
36
37# 2. Create the .env file
38cat > .env << 'EOF'
39# Concourse CI
40CONCOURSE_PORT=8080
41DB_PASSWORD=concourse_password
42ADMIN_PASSWORD=admin_password
43EOF
44
45# 3. Start the services
46docker compose up -d
47
48# 4. View logs
49docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/concourse-ci-stack/run | bash

Troubleshooting

  • Concourse container fails to start with permission errors: Ensure Docker daemon supports privileged containers and the host allows container privilege escalation
  • Pipeline tasks fail with 'no workers available': Restart the concourse container as quickstart mode combines web and worker in single process
  • Fly CLI connection refused: Verify CONCOURSE_EXTERNAL_URL matches your actual access URL and port mapping is correct
  • Resource check failures with Git repositories: Ensure Concourse container has network access to external Git servers and registries
  • Database connection errors on startup: Check that POSTGRES_PASSWORD and DB_PASSWORD environment variables match exactly
  • Pipeline upload fails with authentication errors: Login with fly CLI using admin credentials defined in ADMIN_PASSWORD environment variable

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

concoursepostgresql

Tags

#concourse#ci#pipelines#containers

Category

DevOps & CI/CD

Related Recipes

Ad Space