Redpanda

intermediate

Kafka-compatible streaming platform without Zookeeper.

Overview

Redpanda is a modern streaming data platform that provides a Kafka-compatible API without the operational complexity of traditional Kafka deployments. Written in C++ rather than Java, Redpanda eliminates the need for ZooKeeper, reduces memory overhead, and delivers significantly lower latencies while maintaining full compatibility with existing Kafka clients and tools. This makes it an ideal drop-in replacement for organizations looking to simplify their streaming infrastructure. This deployment configuration launches two services: the core Redpanda broker and the Redpanda Console web interface. The Redpanda service runs as a single-node cluster configured for development and testing, exposing the Kafka API on port 9092 and the HTTP Proxy (Panda Proxy) on port 8082. The Console service provides a user-friendly web interface on port 8080 for managing topics, viewing messages, and monitoring cluster health without requiring separate tooling. This setup is perfect for developers migrating from Kafka who want to experience Redpanda's simplified operations, teams building real-time applications that need lower latency than traditional Kafka, and organizations seeking to reduce their streaming infrastructure footprint. The combination of the broker and console provides everything needed to evaluate Redpanda's performance benefits while maintaining compatibility with existing Kafka-based applications and workflows.

Key Features

100% Kafka API compatibility with existing clients and tools
ZooKeeper-free architecture eliminating coordination overhead
C++-based implementation delivering 10x better performance than JVM-based solutions
Built-in HTTP Proxy for REST API access to streaming data
Single binary deployment reducing operational complexity
Redpanda Console web interface for topic management and monitoring
Tiered storage support for cost-effective data retention
WebAssembly-based stream processing transforms

Common Use Cases

1Kafka migration projects requiring zero application code changes
2Real-time analytics pipelines demanding ultra-low latency processing
3IoT data ingestion where resource efficiency is critical
4Development environments needing simplified streaming infrastructure
5Edge computing deployments with limited operational overhead
6Event-driven microservices architectures requiring reliable message delivery
7Log aggregation systems processing high-volume data streams

Prerequisites

Minimum 2GB RAM available for optimal Redpanda performance
Docker and Docker Compose installed on the host system
Ports 8080, 8082, 9092, and 9644 available on the host
Basic understanding of Kafka concepts like topics and partitions
Familiarity with streaming data patterns and use cases

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

1services: 
2  redpanda: 
3    image: redpandadata/redpanda:latest
4    container_name: redpanda
5    command: 
6      - redpanda start
7      - --smp 1
8      - --memory 1G
9      - --overprovisioned
10      - --node-id 0
11      - --kafka-addr PLAINTEXT://0.0.0.0:9092
12      - --advertise-kafka-addr PLAINTEXT://localhost:9092
13      - --pandaproxy-addr 0.0.0.0:8082
14      - --advertise-pandaproxy-addr localhost:8082
15    volumes: 
16      - redpanda_data:/var/lib/redpanda/data
17    ports: 
18      - "9092:9092"
19      - "8082:8082"
20      - "9644:9644"
21
22  console: 
23    image: redpandadata/console:latest
24    container_name: redpanda-console
25    environment: 
26      KAFKA_BROKERS: redpanda:9092
27    ports: 
28      - "8080:8080"
29    depends_on: 
30      - redpanda
31
32volumes: 
33  redpanda_data:

.env Template

.env

1# No additional config needed

Usage Notes

1Docs: https://docs.redpanda.com/
2Console at http://localhost:8080, Kafka API on port 9092
3100% Kafka API compatible - use existing Kafka clients
4No Zookeeper required - simpler operations
5HTTP Proxy (Panda Proxy) on 8082 for REST API access
6Lower latency than Kafka, written in C++

Individual Services(2 services)

Copy individual services to mix and match with your existing compose files.

redpanda

redpanda:
  image: redpandadata/redpanda:latest
  container_name: redpanda
  command:
    - redpanda start
    - "--smp 1"
    - "--memory 1G"
    - "--overprovisioned"
    - "--node-id 0"
    - "--kafka-addr PLAINTEXT://0.0.0.0:9092"
    - "--advertise-kafka-addr PLAINTEXT://localhost:9092"
    - "--pandaproxy-addr 0.0.0.0:8082"
    - "--advertise-pandaproxy-addr localhost:8082"
  volumes:
    - redpanda_data:/var/lib/redpanda/data
  ports:
    - "9092:9092"
    - "8082:8082"
    - "9644:9644"

console

console:
  image: redpandadata/console:latest
  container_name: redpanda-console
  environment:
    KAFKA_BROKERS: redpanda:9092
  ports:
    - "8080:8080"
  depends_on:
    - redpanda

Quick Start

terminal

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  redpanda:
5    image: redpandadata/redpanda:latest
6    container_name: redpanda
7    command:
8      - redpanda start
9      - --smp 1
10      - --memory 1G
11      - --overprovisioned
12      - --node-id 0
13      - --kafka-addr PLAINTEXT://0.0.0.0:9092
14      - --advertise-kafka-addr PLAINTEXT://localhost:9092
15      - --pandaproxy-addr 0.0.0.0:8082
16      - --advertise-pandaproxy-addr localhost:8082
17    volumes:
18      - redpanda_data:/var/lib/redpanda/data
19    ports:
20      - "9092:9092"
21      - "8082:8082"
22      - "9644:9644"
23
24  console:
25    image: redpandadata/console:latest
26    container_name: redpanda-console
27    environment:
28      KAFKA_BROKERS: redpanda:9092
29    ports:
30      - "8080:8080"
31    depends_on:
32      - redpanda
33
34volumes:
35  redpanda_data:
36EOF
37
38# 2. Create the .env file
39cat > .env << 'EOF'
40# No additional config needed
41EOF
42
43# 3. Start the services
44docker compose up -d
45
46# 4. View logs
47docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal

1curl -fsSL https://docker.recipes/api/recipes/redpanda/run | bash

Troubleshooting

Redpanda fails to start with memory errors: Increase Docker memory limit above 2GB or reduce --memory parameter in the command
Cannot connect to Kafka API on localhost:9092: Verify port 9092 is not in use by another service and check firewall settings
Console shows 'Cannot connect to broker': Ensure redpanda service is healthy and running before console startup
Topics not visible in Console interface: Check KAFKA_BROKERS environment variable points to redpanda:9092 internal address
High CPU usage with --overprovisioned flag: Remove the flag if running on dedicated hardware with sufficient resources
Panda Proxy HTTP API returns connection refused: Verify port 8082 mapping and that pandaproxy-addr is properly configured

Community Notes

Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components