# Redpanda

> Kafka-compatible streaming platform without Zookeeper.

- Category: Message Queues & Brokers
- Difficulty: intermediate
- Components: redpanda
- Tags: redpanda, kafka, streaming, no-zookeeper
- Source: https://docker.recipes/messaging/redpanda

## Overview

Redpanda is a modern streaming data platform that provides a Kafka-compatible API without the operational complexity of traditional Kafka deployments. Written in C++ rather than Java, Redpanda eliminates the need for ZooKeeper, reduces memory overhead, and delivers significantly lower latencies while maintaining full compatibility with existing Kafka clients and tools. This makes it an ideal drop-in replacement for organizations looking to simplify their streaming infrastructure.

This deployment configuration launches two services: the core Redpanda broker and the Redpanda Console web interface. The Redpanda service runs as a single-node cluster configured for development and testing, exposing the Kafka API on port 9092 and the HTTP Proxy (Panda Proxy) on port 8082. The Console service provides a user-friendly web interface on port 8080 for managing topics, viewing messages, and monitoring cluster health without requiring separate tooling.

This setup is perfect for developers migrating from Kafka who want to experience Redpanda's simplified operations, teams building real-time applications that need lower latency than traditional Kafka, and organizations seeking to reduce their streaming infrastructure footprint. The combination of the broker and console provides everything needed to evaluate Redpanda's performance benefits while maintaining compatibility with existing Kafka-based applications and workflows.

## Features

- 100% Kafka API compatibility with existing clients and tools
- ZooKeeper-free architecture eliminating coordination overhead
- C++-based implementation delivering 10x better performance than JVM-based solutions
- Built-in HTTP Proxy for REST API access to streaming data
- Single binary deployment reducing operational complexity
- Redpanda Console web interface for topic management and monitoring
- Tiered storage support for cost-effective data retention
- WebAssembly-based stream processing transforms

## Use cases

- Kafka migration projects requiring zero application code changes
- Real-time analytics pipelines demanding ultra-low latency processing
- IoT data ingestion where resource efficiency is critical
- Development environments needing simplified streaming infrastructure
- Edge computing deployments with limited operational overhead
- Event-driven microservices architectures requiring reliable message delivery
- Log aggregation systems processing high-volume data streams

## Prerequisites

- Minimum 2GB RAM available for optimal Redpanda performance
- Docker and Docker Compose installed on the host system
- Ports 8080, 8082, 9092, and 9644 available on the host
- Basic understanding of Kafka concepts like topics and partitions
- Familiarity with streaming data patterns and use cases

## docker-compose.yml

```yaml
services:
  redpanda:
    image: redpandadata/redpanda:latest
    container_name: redpanda
    command:
      - redpanda start
      - --smp 1
      - --memory 1G
      - --overprovisioned
      - --node-id 0
      - --kafka-addr PLAINTEXT://0.0.0.0:9092
      - --advertise-kafka-addr PLAINTEXT://localhost:9092
      - --pandaproxy-addr 0.0.0.0:8082
      - --advertise-pandaproxy-addr localhost:8082
    volumes:
      - redpanda_data:/var/lib/redpanda/data
    ports:
      - "9092:9092"
      - "8082:8082"
      - "9644:9644"

  console:
    image: redpandadata/console:latest
    container_name: redpanda-console
    environment:
      KAFKA_BROKERS: redpanda:9092
    ports:
      - "8080:8080"
    depends_on:
      - redpanda

volumes:
  redpanda_data:
```

## .env

```bash
# No additional config needed
```

## Usage notes

- Docs: https://docs.redpanda.com/
- Console at http://localhost:8080, Kafka API on port 9092
- 100% Kafka API compatible - use existing Kafka clients
- No Zookeeper required - simpler operations
- HTTP Proxy (Panda Proxy) on 8082 for REST API access
- Lower latency than Kafka, written in C++

## Troubleshooting

- Redpanda fails to start with memory errors: Increase Docker memory limit above 2GB or reduce --memory parameter in the command
- Cannot connect to Kafka API on localhost:9092: Verify port 9092 is not in use by another service and check firewall settings
- Console shows 'Cannot connect to broker': Ensure redpanda service is healthy and running before console startup
- Topics not visible in Console interface: Check KAFKA_BROKERS environment variable points to redpanda:9092 internal address
- High CPU usage with --overprovisioned flag: Remove the flag if running on dedicated hardware with sufficient resources
- Panda Proxy HTTP API returns connection refused: Verify port 8082 mapping and that pandaproxy-addr is properly configured

