# Apache Pulsar Cluster

> Apache Pulsar messaging and streaming platform with manager UI.

- Category: Message Queues & Brokers
- Difficulty: advanced
- Components: pulsar, pulsar-manager
- Tags: pulsar, messaging, streaming, pub-sub, distributed
- Source: https://docker.recipes/messaging/apache-pulsar-cluster

## Overview

Apache Pulsar is a cloud-native distributed messaging and event-streaming platform originally developed at Yahoo and later open-sourced to the Apache Software Foundation. Unlike traditional messaging systems, Pulsar combines the best of both worlds by providing unified queuing and streaming capabilities in a single platform, with built-in multi-tenancy, geo-replication, and tiered storage. Its unique architecture separates computation from storage, using Apache BookKeeper as the underlying storage layer, which enables infinite message retention and horizontal scaling without data rebalancing. Pulsar Manager serves as a comprehensive web-based administration interface that simplifies cluster management, tenant configuration, and monitoring. Together, they form a complete messaging infrastructure that handles everything from real-time stream processing to traditional message queuing, with advanced features like schema evolution, transactions, and Pulsar Functions for serverless computing. This combination is particularly valuable for organizations building modern, cloud-native applications that require reliable, scalable messaging with enterprise-grade features like multi-tenancy and disaster recovery through geo-replication.

## Features

- Unified queuing and streaming model supporting both pub-sub and message queuing patterns
- Multi-tenant architecture with namespace isolation and resource quotas
- Geo-replication for disaster recovery and global message distribution
- Tiered storage for automatic data archiving to cloud storage systems
- Built-in schema registry with schema evolution support
- Pulsar Functions for serverless stream processing and transformations
- Topic compaction for maintaining latest state per message key
- Web-based Pulsar Manager UI for cluster administration and monitoring

## Use cases

- Event-driven microservices architectures requiring reliable message delivery
- Multi-tenant SaaS platforms needing isolated messaging per customer
- IoT data ingestion and real-time analytics for sensor networks
- Financial trading systems requiring low-latency message processing
- Log aggregation and centralized logging for distributed applications
- Stream processing pipelines for real-time data transformation
- Cross-datacenter replication for global application deployments

## Prerequisites

- Minimum 4GB RAM recommended for standalone Pulsar deployment
- Docker Engine 19.03 or higher with Docker Compose v2 support
- Ports 6650, 8080, 9527, and 7750 available on the host system
- Understanding of messaging patterns and pub-sub concepts
- Basic knowledge of Apache BookKeeper storage concepts
- Familiarity with tenant and namespace management in multi-tenant systems

## docker-compose.yml

```yaml
services:
  pulsar:
    image: apachepulsar/pulsar:latest
    container_name: pulsar
    restart: unless-stopped
    command: bin/pulsar standalone
    ports:
      - "${PULSAR_PORT:-6650}:6650"
      - "${PULSAR_HTTP_PORT:-8080}:8080"
    volumes:
      - pulsar_data:/pulsar/data
      - pulsar_conf:/pulsar/conf
    networks:
      - pulsar-network

  pulsar-manager:
    image: apachepulsar/pulsar-manager:latest
    container_name: pulsar-manager
    restart: unless-stopped
    ports:
      - "${MANAGER_PORT:-9527}:9527"
      - "7750:7750"
    depends_on:
      - pulsar
    environment:
      SPRING_CONFIGURATION_FILE: /pulsar-manager/pulsar-manager/application.properties
    networks:
      - pulsar-network

volumes:
  pulsar_data:
  pulsar_conf:

networks:
  pulsar-network:
    driver: bridge
```

## .env

```bash
# Apache Pulsar
PULSAR_PORT=6650
PULSAR_HTTP_PORT=8080
MANAGER_PORT=9527
```

## Usage notes

- Pulsar broker at localhost:6650
- Admin API at http://localhost:8080
- Pulsar Manager at http://localhost:9527
- Multi-tenant messaging system

## Troubleshooting

- Pulsar Manager shows 'Connection refused' to broker: Ensure pulsar service is fully started and accessible on port 6650 before accessing manager UI
- Out of memory errors during startup: Increase Docker memory allocation to at least 4GB or set PULSAR_MEM environment variable
- Topics not visible in Pulsar Manager: Initialize manager with proper cluster configuration using the REST API at port 7750
- Persistent volume permission denied errors: Ensure Docker has proper permissions to write to mounted volume directories
- Slow message consumption: Check if consumer is acknowledging messages properly and consider adjusting receive queue size
- BookKeeper write failures: Verify sufficient disk space and that ensemble/quorum settings match available bookies

