MLflow

intermediate

ML lifecycle management including tracking and model registry.

Overview

MLflow is an open-source machine learning lifecycle management platform developed by Databricks that addresses the challenge of tracking experiments, managing models, and deploying ML solutions at scale. Originally created to solve the fragmented nature of ML workflows across different frameworks and tools, MLflow provides a unified interface for experiment tracking, model versioning, and deployment orchestration. The platform supports multiple ML libraries including scikit-learn, TensorFlow, PyTorch, and XGBoost, making it framework-agnostic for diverse ML teams. This MLflow deployment combines the MLflow tracking server with PostgreSQL as the backend metadata store, creating a robust production-ready ML operations environment. PostgreSQL stores all experiment metadata, parameters, metrics, and model registry information, while MLflow manages artifact storage and provides the web UI for visualization and model management. The combination offers superior data integrity and query performance compared to SQLite-based deployments, supporting concurrent users and complex experiment queries. Data scientists and ML engineers working in team environments benefit most from this stack, particularly those managing multiple experiments, model versions, and deployment stages. Organizations implementing MLOps practices will find this configuration essential for maintaining experiment reproducibility, model governance, and collaboration across ML teams. The PostgreSQL backend ensures enterprise-grade reliability for storing critical ML metadata and enables advanced querying of experiment results across projects and teams.

Key Features

Experiment tracking with automatic parameter and metric logging across ML frameworks
Model registry with versioning, stage transitions (staging/production), and lineage tracking
PostgreSQL backend for enterprise-grade metadata storage with ACID compliance
Artifact storage management for models, plots, and datasets with version control
Multi-framework support including scikit-learn, TensorFlow, PyTorch, and XGBoost
REST API and Python SDK for programmatic experiment and model management
Web UI for experiment comparison, model visualization, and team collaboration
Model serving capabilities with automatic endpoint generation from registered models

Common Use Cases

1ML teams tracking hyperparameter tuning experiments across multiple models and datasets
2Data science organizations implementing model governance and approval workflows
3Research teams comparing algorithm performance across different experimental conditions
4MLOps pipelines requiring automated model versioning and deployment stage management
5Enterprise ML platforms centralizing experiment tracking across multiple data science teams
6Model validation workflows tracking A/B testing results and production model performance
7Academic research environments managing reproducible ML experiments and publications

Prerequisites

Minimum 1.5GB RAM for combined MLflow and PostgreSQL operations
Port 5000 available for MLflow web interface access
Docker environment variables configured: DB_NAME, DB_USER, DB_PASSWORD
Basic understanding of ML experiment tracking concepts and model lifecycle management
Python environment with MLflow client library for programmatic access
Storage space for ML artifacts including model files, plots, and datasets

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

1services: 
2  mlflow: 
3    image: ghcr.io/mlflow/mlflow:latest
4    container_name: mlflow
5    restart: unless-stopped
6    command: mlflow server --backend-store-uri postgresql://${DB_USER}:${DB_PASSWORD}@postgres:5432/${DB_NAME} --default-artifact-root /mlflow/artifacts --host 0.0.0.0
7    volumes: 
8      - mlflow_artifacts:/mlflow/artifacts
9    ports: 
10      - "5000:5000"
11    depends_on: 
12      - postgres
13    networks: 
14      - mlflow
15
16  postgres: 
17    image: postgres:16-alpine
18    container_name: mlflow-postgres
19    environment: 
20      POSTGRES_DB: ${DB_NAME}
21      POSTGRES_USER: ${DB_USER}
22      POSTGRES_PASSWORD: ${DB_PASSWORD}
23    volumes: 
24      - postgres_data:/var/lib/postgresql/data
25    networks: 
26      - mlflow
27
28volumes: 
29  mlflow_artifacts: 
30  postgres_data: 
31
32networks: 
33  mlflow: 
34    driver: bridge

.env Template

.env

1DB_NAME=mlflow
2DB_USER=mlflow
3DB_PASSWORD=changeme

Usage Notes

1Docs: https://mlflow.org/docs/latest/index.html
2Access UI at http://localhost:5000 - no auth by default
3Track experiments: mlflow.log_param(), mlflow.log_metric()
4Model registry for versioning and staging (staging/production)
5Artifacts stored at /mlflow/artifacts, metadata in PostgreSQL
6Python SDK: pip install mlflow, then mlflow.set_tracking_uri()

Individual Services(2 services)

Copy individual services to mix and match with your existing compose files.

mlflow

mlflow:
  image: ghcr.io/mlflow/mlflow:latest
  container_name: mlflow
  restart: unless-stopped
  command: mlflow server --backend-store-uri postgresql://${DB_USER}:${DB_PASSWORD}@postgres:5432/${DB_NAME} --default-artifact-root /mlflow/artifacts --host 0.0.0.0
  volumes:
    - mlflow_artifacts:/mlflow/artifacts
  ports:
    - "5000:5000"
  depends_on:
    - postgres
  networks:
    - mlflow

postgres

postgres:
  image: postgres:16-alpine
  container_name: mlflow-postgres
  environment:
    POSTGRES_DB: ${DB_NAME}
    POSTGRES_USER: ${DB_USER}
    POSTGRES_PASSWORD: ${DB_PASSWORD}
  volumes:
    - postgres_data:/var/lib/postgresql/data
  networks:
    - mlflow

Quick Start

terminal

1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  mlflow:
5    image: ghcr.io/mlflow/mlflow:latest
6    container_name: mlflow
7    restart: unless-stopped
8    command: mlflow server --backend-store-uri postgresql://${DB_USER}:${DB_PASSWORD}@postgres:5432/${DB_NAME} --default-artifact-root /mlflow/artifacts --host 0.0.0.0
9    volumes:
10      - mlflow_artifacts:/mlflow/artifacts
11    ports:
12      - "5000:5000"
13    depends_on:
14      - postgres
15    networks:
16      - mlflow
17
18  postgres:
19    image: postgres:16-alpine
20    container_name: mlflow-postgres
21    environment:
22      POSTGRES_DB: ${DB_NAME}
23      POSTGRES_USER: ${DB_USER}
24      POSTGRES_PASSWORD: ${DB_PASSWORD}
25    volumes:
26      - postgres_data:/var/lib/postgresql/data
27    networks:
28      - mlflow
29
30volumes:
31  mlflow_artifacts:
32  postgres_data:
33
34networks:
35  mlflow:
36    driver: bridge
37EOF
38
39# 2. Create the .env file
40cat > .env << 'EOF'
41DB_NAME=mlflow
42DB_USER=mlflow
43DB_PASSWORD=changeme
44EOF
45
46# 3. Start the services
47docker compose up -d
48
49# 4. View logs
50docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal

1curl -fsSL https://docker.recipes/api/recipes/mlflow/run | bash

Troubleshooting

MLflow UI shows 'Backend store unavailable': Verify PostgreSQL container is running and database credentials match environment variables
Experiments not appearing in UI: Check MLflow tracking URI configuration in client code matches server address (http://localhost:5000)
Artifact upload failures: Ensure mlflow_artifacts volume has sufficient disk space and proper write permissions
Database connection timeout errors: Increase PostgreSQL shared_buffers and max_connections for high-concurrency experiment logging
Model registry operations failing: Verify PostgreSQL schema migrations completed successfully by checking MLflow server startup logs
Slow experiment query performance: Add database indexes on frequently queried experiment parameters and metrics columns

Community Notes

Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

mlflowpostgres

Related Recipes

Ad Space

Shortcuts: C CopyF FavoriteD Download