# ZeroMQ (with Python)

> High-performance messaging library development setup.

- Category: Message Queues & Brokers
- Difficulty: intermediate
- Components: python
- Tags: zeromq, messaging, sockets, library
- Source: https://docker.recipes/messaging/zeromq

## Overview

ZeroMQ is a high-performance asynchronous messaging library that provides a message queue without requiring a dedicated message broker. Originally developed by iMatix in 2007, ZeroMQ implements socket-like interfaces for various messaging patterns including request-reply, publish-subscribe, and pipeline architectures. Unlike traditional message queuing systems, ZeroMQ operates in a brokerless fashion, establishing direct connections between distributed applications while maintaining ultra-low latency and high throughput capabilities. Python integration through the pyzmq library brings ZeroMQ's powerful messaging capabilities to Python applications, enabling developers to build distributed systems, real-time data pipelines, and concurrent processing applications. The combination leverages Python's rapid development capabilities with ZeroMQ's battle-tested messaging patterns, making it ideal for building scalable microservices, IoT data collection systems, and high-frequency trading platforms. This development environment is particularly valuable for financial technology companies building low-latency trading systems, IoT platform developers managing thousands of sensor connections, and research institutions processing real-time scientific data streams. The brokerless architecture eliminates single points of failure while Python's extensive ecosystem provides rich libraries for data processing, analysis, and machine learning integration.

## Features

- Multiple messaging patterns including REQ/REP, PUB/SUB, PUSH/PULL, and DEALER/ROUTER for different communication scenarios
- Brokerless architecture eliminating message broker dependencies and single points of failure
- Sub-millisecond message latency with throughput capabilities exceeding millions of messages per second
- Built-in load balancing and failover mechanisms for PUSH/PULL and DEALER/ROUTER patterns
- Transport agnostic design supporting TCP, IPC, inproc, and multicast protocols
- Automatic message queuing and buffering with configurable high water marks
- Zero-copy message passing optimization for large payloads and high-throughput scenarios
- Language-agnostic wire protocol enabling polyglot distributed system architectures

## Use cases

- Real-time financial trading platforms requiring sub-millisecond order execution and market data distribution
- IoT sensor networks collecting and aggregating data from thousands of distributed devices
- Distributed computing clusters coordinating parallel processing tasks across multiple worker nodes
- Real-time chat applications and notification systems with instant message delivery
- Scientific computing environments processing streaming data from instruments and simulations
- Microservices architectures requiring high-performance inter-service communication
- Game server backends managing real-time multiplayer state synchronization and player communications

## Prerequisites

- Docker Engine 20.10+ and Docker Compose V2 for container orchestration
- Minimum 512MB RAM allocated to Docker for Python runtime and ZeroMQ library compilation
- Basic understanding of Python package management and virtual environments
- Familiarity with socket programming concepts and network communication patterns
- Knowledge of concurrency models including threading, multiprocessing, and async/await patterns
- Understanding of distributed systems concepts like message queuing and event-driven architectures

## docker-compose.yml

```yaml
services:
  zeromq-dev:
    image: python:3.11-slim
    container_name: zeromq-dev
    command: tail -f /dev/null
    volumes:
      - ./app:/app
    working_dir: /app
```

## .env

```bash
# pip install pyzmq in container
```

## Usage notes

- Docs: https://zeromq.org/get-started/
- Development container for ZeroMQ projects
- Install: pip install pyzmq (Python), npm install zeromq (Node)
- Patterns: REQ/REP (request-reply), PUB/SUB, PUSH/PULL, DEALER/ROUTER
- Brokerless architecture - direct socket connections
- Ultra-low latency - ideal for distributed computing

## Troubleshooting

- ImportError: No module named 'zmq': Install pyzmq using pip install pyzmq or ensure the package is included in requirements.txt
- zmq.error.ZMQError: Address already in use: Change the port number in bind() calls or use zmq.REUSEADDR socket option
- Messages not being delivered in PUB/SUB pattern: Add sleep delay after socket creation to allow subscription propagation or use zmq.CONFLATE for latest message only
- High memory usage with large message queues: Set zmq.HWM (High Water Mark) to limit queue size and implement flow control
- Connection timeouts in distributed environments: Increase zmq.LINGER and zmq.RCVTIMEO socket options or implement heartbeat mechanisms
- Segmentation faults during container shutdown: Properly close sockets and terminate ZMQ context using context.term() before process exit

