Apache Pulsar

advanced

Distributed messaging and streaming platform.

Overview

Apache Pulsar is a cloud-native, distributed messaging and event-streaming platform originally developed by Yahoo and later open-sourced in 2016. Unlike traditional messaging systems, Pulsar provides a unified approach to both queuing and streaming, supporting multi-tenancy, geo-replication, and tiered storage natively. Its architecture separates compute from storage, allowing for independent scaling and better resource utilization compared to other messaging platforms. This deployment runs Pulsar in standalone mode, which packages all Pulsar components (broker, bookkeeper, and zookeeper) into a single container for development and testing purposes. The standalone configuration provides access to both the binary protocol on port 6650 for client connections and the admin REST API on port 8080 for management operations. This setup is ideal for developers and teams who need to quickly spin up a Pulsar instance for development, testing, or proof-of-concept work without the complexity of managing a full distributed cluster. The standalone mode retains all of Pulsar's core features including multi-tenancy, schema registry, and Pulsar Functions, making it perfect for learning Pulsar's capabilities or building applications that will eventually scale to production clusters.

Key Features

Unified messaging model supporting both pub-sub and queuing semantics
Built-in multi-tenancy with namespace isolation and resource quotas
Native geo-replication for disaster recovery and global distribution
Schema registry with automatic schema evolution and compatibility checking
Pulsar Functions for lightweight stream processing and event handling
Tiered storage support for automatic data archiving to cloud storage
Topic compaction for maintaining latest values of keyed messages
Transaction support for exactly-once processing guarantees

Common Use Cases

1Event-driven microservices architecture requiring reliable message delivery
2Real-time analytics pipelines processing streaming data from IoT devices
3Multi-tenant SaaS platforms needing isolated messaging per customer
4Financial systems requiring exactly-once processing with transaction support
5Global applications needing geo-replicated messaging across regions
6Development and testing environments for Pulsar-based applications
7Stream processing workflows combining messaging with lightweight compute functions

Prerequisites

Minimum 1GB RAM available, 4GB+ recommended for production workloads
Ports 6650 and 8080 available for Pulsar binary protocol and admin API
Understanding of pub-sub messaging patterns and streaming concepts
Basic knowledge of multi-tenancy concepts (tenants, namespaces, topics)
Familiarity with Pulsar client libraries for your preferred programming language
Docker environment with volume support for persistent message storage

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

1services: 
2  pulsar: 
3    image: apachepulsar/pulsar:latest
4    container_name: pulsar
5    restart: unless-stopped
6    command: bin/pulsar standalone
7    volumes: 
8      - pulsar_data:/pulsar/data
9    ports: 
10      - "6650:6650"
11      - "8080:8080"
12
13volumes: 
14  pulsar_data:

.env Template

.env

1# No additional config needed

Usage Notes

1Docs: https://pulsar.apache.org/docs/
2Admin REST API at http://localhost:8080, binary protocol on 6650
3Standalone mode for development - use cluster for production
4Create tenant: pulsar-admin tenants create my-tenant
5Multi-tenancy, geo-replication, tiered storage built-in
6Supports both queuing and streaming semantics

Quick Start

terminal

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  pulsar:
5    image: apachepulsar/pulsar:latest
6    container_name: pulsar
7    restart: unless-stopped
8    command: bin/pulsar standalone
9    volumes:
10      - pulsar_data:/pulsar/data
11    ports:
12      - "6650:6650"
13      - "8080:8080"
14
15volumes:
16  pulsar_data:
17EOF
18
19# 2. Create the .env file
20cat > .env << 'EOF'
21# No additional config needed
22EOF
23
24# 3. Start the services
25docker compose up -d
26
27# 4. View logs
28docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal

1curl -fsSL https://docker.recipes/api/recipes/pulsar/run | bash

Troubleshooting

Container fails to start with 'Not enough memory' error: Increase Docker memory allocation to at least 1GB or add JVM memory limits to the command
Client connection refused on port 6650: Verify the container is fully started by checking logs for 'Pulsar Standalone is running' message
Admin API returns 404 errors: Ensure you're using the correct REST endpoint format like http://localhost:8080/admin/v2/tenants
Topics not found errors: Create tenant and namespace first using pulsar-admin commands before creating topics
Persistent storage issues: Verify the pulsar_data volume is properly mounted and has sufficient disk space
Schema registry errors: Check that schemas are properly registered and compatible when producing/consuming messages

Community Notes

Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

pulsar

Related Recipes

Ad Space

Shortcuts: C CopyF FavoriteD Download