docker.recipes

K3s Development Cluster

advanced

Lightweight Kubernetes with k3s, Traefik ingress, and local registry for development.

Overview

K3s is Rancher's production-ready, CNCF-certified Kubernetes distribution designed for resource-constrained environments and edge computing. Built from the ground up to be lightweight, K3s removes optional and legacy features from standard Kubernetes, packages everything into a single binary under 100MB, and replaces etcd with SQLite by default. This makes it perfect for development environments, IoT deployments, ARM devices, and situations where full Kubernetes would be overkill. This deployment creates a complete K3s development cluster using three specialized containers: a k3s-server running the full Kubernetes control plane and worker node in a single process, a Docker Registry v2 for storing and distributing container images locally, and a web-based registry UI for browsing and managing those images. The k3s-server includes Traefik as the default ingress controller, providing automatic load balancing and SSL termination capabilities built into the cluster. The configuration automatically sets up kubeconfig file generation and configures the cluster to use the local registry for pulling images. This stack is ideal for developers who need a full-featured Kubernetes environment for testing applications, CI/CD pipelines that require isolated Kubernetes clusters, and teams wanting to prototype cloud-native applications without the complexity and resource overhead of full Kubernetes distributions like kubeadm or managed services. The local registry integration eliminates the need to push images to external registries during development, significantly speeding up the container build-test-deploy cycle.

Key Features

  • Complete Kubernetes API compatibility in a single lightweight container under 100MB
  • Built-in Traefik ingress controller with automatic service discovery and load balancing
  • Local Docker Registry v2 with pre-configured integration for seamless image management
  • Web-based registry UI for browsing, searching, and managing container images and tags
  • Automatic kubeconfig generation with external access via port 6443
  • SQLite-backed storage eliminating etcd complexity and resource requirements
  • Privileged container mode enabling full Kubernetes networking and storage features
  • Pre-configured registries.yaml for automatic local registry authentication

Common Use Cases

  • 1Local Kubernetes development environment for testing microservices and cloud-native applications
  • 2CI/CD pipeline testing requiring isolated, disposable Kubernetes clusters
  • 3Kubernetes training and certification labs with realistic cluster behavior
  • 4Edge computing prototypes requiring lightweight Kubernetes at remote locations
  • 5Container image development workflow with local registry for rapid iteration
  • 6Helm chart development and testing without external cluster dependencies
  • 7GitOps and ArgoCD testing in controlled local environment

Prerequisites

  • Docker Engine with privileged container support enabled
  • Minimum 2GB RAM available (4GB+ recommended for running workloads)
  • Ports 6443, 80, 443, 5000, and 8080 available on host system
  • Basic Kubernetes and kubectl knowledge for cluster interaction
  • Understanding of container registries and image tagging conventions
  • K3S_TOKEN environment variable set for cluster security

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  k3s-server: 
3    image: rancher/k3s:latest
4    command: server
5    tmpfs: 
6      - /run
7      - /var/run
8    ulimits: 
9      nproc: 65535
10      nofile: 
11        soft: 65535
12        hard: 65535
13    privileged: true
14    environment: 
15      K3S_TOKEN: ${K3S_TOKEN}
16      K3S_KUBECONFIG_OUTPUT: /output/kubeconfig.yaml
17      K3S_KUBECONFIG_MODE: 666
18    volumes: 
19      - k3s_server:/var/lib/rancher/k3s
20      - ./output:/output
21      - ./registries.yaml:/etc/rancher/k3s/registries.yaml:ro
22    ports: 
23      - "6443:6443"
24      - "80:80"
25      - "443:443"
26    networks: 
27      - k3s-net
28    restart: unless-stopped
29
30  registry: 
31    image: registry:2
32    ports: 
33      - "5000:5000"
34    volumes: 
35      - registry_data:/var/lib/registry
36    networks: 
37      - k3s-net
38    restart: unless-stopped
39
40  registry-ui: 
41    image: joxit/docker-registry-ui:latest
42    ports: 
43      - "8080:80"
44    environment: 
45      REGISTRY_TITLE: Local Registry
46      REGISTRY_URL: http://registry:5000
47      SINGLE_REGISTRY: "true"
48    depends_on: 
49      - registry
50    networks: 
51      - k3s-net
52    restart: unless-stopped
53
54volumes: 
55  k3s_server: 
56  registry_data: 
57
58networks: 
59  k3s-net: 
60    driver: bridge

.env Template

.env
1# K3s Token
2K3S_TOKEN=your_secret_token_here
3
4# Generate with: openssl rand -hex 32

Usage Notes

  1. 1Kubeconfig at ./output/kubeconfig.yaml
  2. 2Local registry at localhost:5000
  3. 3Registry UI at http://localhost:8080
  4. 4Tag images: localhost:5000/myapp:latest

Individual Services(3 services)

Copy individual services to mix and match with your existing compose files.

k3s-server
k3s-server:
  image: rancher/k3s:latest
  command: server
  tmpfs:
    - /run
    - /var/run
  ulimits:
    nproc: 65535
    nofile:
      soft: 65535
      hard: 65535
  privileged: true
  environment:
    K3S_TOKEN: ${K3S_TOKEN}
    K3S_KUBECONFIG_OUTPUT: /output/kubeconfig.yaml
    K3S_KUBECONFIG_MODE: 666
  volumes:
    - k3s_server:/var/lib/rancher/k3s
    - ./output:/output
    - ./registries.yaml:/etc/rancher/k3s/registries.yaml:ro
  ports:
    - "6443:6443"
    - "80:80"
    - "443:443"
  networks:
    - k3s-net
  restart: unless-stopped
registry
registry:
  image: registry:2
  ports:
    - "5000:5000"
  volumes:
    - registry_data:/var/lib/registry
  networks:
    - k3s-net
  restart: unless-stopped
registry-ui
registry-ui:
  image: joxit/docker-registry-ui:latest
  ports:
    - "8080:80"
  environment:
    REGISTRY_TITLE: Local Registry
    REGISTRY_URL: http://registry:5000
    SINGLE_REGISTRY: "true"
  depends_on:
    - registry
  networks:
    - k3s-net
  restart: unless-stopped

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  k3s-server:
5    image: rancher/k3s:latest
6    command: server
7    tmpfs:
8      - /run
9      - /var/run
10    ulimits:
11      nproc: 65535
12      nofile:
13        soft: 65535
14        hard: 65535
15    privileged: true
16    environment:
17      K3S_TOKEN: ${K3S_TOKEN}
18      K3S_KUBECONFIG_OUTPUT: /output/kubeconfig.yaml
19      K3S_KUBECONFIG_MODE: 666
20    volumes:
21      - k3s_server:/var/lib/rancher/k3s
22      - ./output:/output
23      - ./registries.yaml:/etc/rancher/k3s/registries.yaml:ro
24    ports:
25      - "6443:6443"
26      - "80:80"
27      - "443:443"
28    networks:
29      - k3s-net
30    restart: unless-stopped
31
32  registry:
33    image: registry:2
34    ports:
35      - "5000:5000"
36    volumes:
37      - registry_data:/var/lib/registry
38    networks:
39      - k3s-net
40    restart: unless-stopped
41
42  registry-ui:
43    image: joxit/docker-registry-ui:latest
44    ports:
45      - "8080:80"
46    environment:
47      REGISTRY_TITLE: Local Registry
48      REGISTRY_URL: http://registry:5000
49      SINGLE_REGISTRY: "true"
50    depends_on:
51      - registry
52    networks:
53      - k3s-net
54    restart: unless-stopped
55
56volumes:
57  k3s_server:
58  registry_data:
59
60networks:
61  k3s-net:
62    driver: bridge
63EOF
64
65# 2. Create the .env file
66cat > .env << 'EOF'
67# K3s Token
68K3S_TOKEN=your_secret_token_here
69
70# Generate with: openssl rand -hex 32
71EOF
72
73# 3. Start the services
74docker compose up -d
75
76# 4. View logs
77docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/k3s-development-cluster/run | bash

Troubleshooting

  • k3s-server fails to start with 'permission denied': Ensure Docker daemon has privileged container support and the container is running in privileged mode
  • kubectl connection refused on port 6443: Check that ./output directory exists with proper permissions and kubeconfig.yaml is generated
  • Images fail to pull from localhost:5000: Verify registries.yaml configuration is mounted correctly and registry service is running
  • registry-ui shows 'registry not found' error: Confirm registry container is healthy and accessible on the k3s-net network
  • Pods stuck in Pending state: Check if k3s-server has sufficient resources and tmpfs mounts are properly configured
  • Traefik ingress not routing traffic: Ensure ports 80/443 are not blocked by host firewall and IngressRoute resources are properly configured

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

k3sregistrytraefik

Tags

#k3s#kubernetes#development#traefik#registry

Category

Development Tools

Related Recipes

Ad Space