# ClearML

> ML experiment tracking and automation platform.

- Category: AI & Machine Learning
- Difficulty: intermediate
- Components: clearml, elasticsearch, mongo, redis
- Tags: clearml, mlops, tracking, experiments
- Source: https://docker.recipes/ai-ml/clearml

## Overview

ClearML is an open-source MLOps platform that automates machine learning experiment tracking, model management, and workflow orchestration. Originally developed by Allegro AI, ClearML addresses the critical challenge of reproducibility in machine learning by automatically capturing experiment parameters, metrics, and artifacts without requiring code changes. It integrates with popular ML frameworks like PyTorch, TensorFlow, and scikit-learn to provide comprehensive experiment lineage and model versioning capabilities.

This stack combines ClearML's three core services with a robust data infrastructure: MongoDB stores experiment metadata and project configurations, Elasticsearch indexes and searches through experiment logs and metrics for fast retrieval, and Redis provides high-speed caching for real-time experiment monitoring and task queuing. The architecture separates concerns with dedicated containers for the web interface, API server, and file storage, enabling scalable deployment and efficient resource utilization.

Data scientists and ML engineers working on teams will find this configuration invaluable for establishing experiment reproducibility, comparing model performance across iterations, and automating ML pipelines. Organizations transitioning from ad-hoc ML development to structured MLOps practices benefit from ClearML's ability to retrofit existing codebases with tracking capabilities, while the included data stack ensures reliable storage and fast querying of experiment history even as projects scale to thousands of experiments.

## Features

- Automatic experiment tracking for PyTorch, TensorFlow, Keras, and scikit-learn without code modifications
- Web-based experiment comparison with interactive plots and hyperparameter visualization
- MongoDB-backed experiment metadata storage with flexible schema for diverse ML frameworks
- Elasticsearch-powered full-text search across experiment logs, parameters, and model descriptions
- Redis-accelerated real-time experiment monitoring and distributed task queue management
- File server for artifact storage including model checkpoints, datasets, and visualization assets
- REST API server enabling programmatic access to experiment data and remote task execution
- Agent-based remote execution system for distributed training and hyperparameter optimization

## Use cases

- ML research teams tracking hundreds of hyperparameter tuning experiments across multiple models
- Data science organizations implementing experiment reproducibility standards and audit trails
- Distributed machine learning workflows requiring remote agent execution on GPU clusters
- Model comparison and selection processes with automated metric tracking and visualization
- MLOps pipeline automation with programmatic experiment triggering and result analysis
- Academic research groups needing collaborative experiment sharing and progress tracking
- Production ML teams managing model versioning and deployment artifact lineage

## Prerequisites

- Minimum 6GB RAM (2GB for Elasticsearch, 2GB for MongoDB, 1GB for ClearML services, 1GB system overhead)
- Docker and Docker Compose with support for multi-container networking and volume persistence
- Ports 8080, 8008, and 8081 available for ClearML web interface, API server, and file server
- Understanding of machine learning workflows and experiment tracking concepts
- Python development environment for ClearML SDK installation and configuration
- Sufficient disk space for experiment artifacts, model checkpoints, and database storage (recommend 50GB+)

## docker-compose.yml

```yaml
services:
  clearml-webserver:
    image: allegroai/clearml:latest
    container_name: clearml-webserver
    restart: unless-stopped
    environment:
      CLEARML_HOST_IP: localhost
      CLEARML__apiserver__mongo__host: mongo
      CLEARML__apiserver__es__host: elasticsearch
    ports:
      - "8080:8080"
    depends_on:
      - mongo
      - elasticsearch
      - redis
    networks:
      - clearml

  clearml-apiserver:
    image: allegroai/clearml:latest
    container_name: clearml-apiserver
    restart: unless-stopped
    command: apiserver
    environment:
      CLEARML__apiserver__mongo__host: mongo
      CLEARML__apiserver__es__host: elasticsearch
      CLEARML__redis__host: redis
    ports:
      - "8008:8008"
    depends_on:
      - mongo
      - elasticsearch
      - redis
    networks:
      - clearml

  clearml-fileserver:
    image: allegroai/clearml:latest
    container_name: clearml-fileserver
    command: fileserver
    ports:
      - "8081:8081"
    volumes:
      - clearml_files:/mnt/fileserver
    networks:
      - clearml

  mongo:
    image: mongo:4.4
    container_name: clearml-mongo
    volumes:
      - mongo_data:/data/db
    networks:
      - clearml

  elasticsearch:
    image: elasticsearch:7.17.0
    container_name: clearml-es
    environment:
      - discovery.type=single-node
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
    volumes:
      - es_data:/usr/share/elasticsearch/data
    networks:
      - clearml

  redis:
    image: redis:alpine
    container_name: clearml-redis
    networks:
      - clearml

volumes:
  clearml_files:
  mongo_data:
  es_data:

networks:
  clearml:
    driver: bridge
```

## .env

```bash
# Configure credentials via web UI
```

## Usage notes

- Docs: https://clear.ml/docs/latest/
- Web UI at http://localhost:8080 - configure credentials on first visit
- API server at http://localhost:8008, file server at :8081
- Python SDK: pip install clearml, then clearml-init to configure
- Auto-logs PyTorch, TensorFlow, scikit-learn experiments
- Agent for remote execution: clearml-agent daemon --queue default

## Troubleshooting

- ClearML web interface shows 'Server not responding': Verify clearml-apiserver container is running and MongoDB connection is established
- Elasticsearch container exits with 'max virtual memory areas vm.max_map_count too low': Run 'sysctl -w vm.max_map_count=262144' on Docker host
- Experiment tracking fails with 'Could not connect to API server': Ensure port 8008 is accessible and run 'clearml-init' to configure SDK credentials
- MongoDB connection errors in API server logs: Check that mongo container is fully started before clearml-apiserver attempts connection
- File upload failures to clearml-fileserver: Verify clearml_files volume has proper write permissions and sufficient disk space
- Redis connection timeouts during high experiment load: Increase Redis memory limit or enable Redis persistence for data durability

