# Ollama with Open WebUI

> Ollama local LLMs with Open WebUI chat.

- Category: AI & Machine Learning
- Difficulty: beginner
- Components: ollama, open-webui
- Tags: ollama, llm, chat, local-ai
- Source: https://docker.recipes/ai-ml/ollama-webui-stack

## Overview

Ollama is a revolutionary tool that democratizes access to large language models by enabling users to run powerful AI models like Llama 2, Mistral, and Code Llama directly on their local machines. Created to address the privacy concerns and API costs associated with cloud-based AI services, Ollama provides a simple command-line interface for downloading, managing, and serving LLMs with OpenAI-compatible APIs. Its efficient model quantization and GPU acceleration capabilities make it possible to run sophisticated AI models on consumer hardware.

This stack combines Ollama's local LLM serving capabilities with Open WebUI's intuitive chat interface, creating a complete self-hosted AI chat solution. Open WebUI transforms Ollama's command-line functionality into a familiar ChatGPT-like web interface, complete with conversation history, model switching, and multi-user support. The integration allows users to interact with their locally-hosted models through a polished web interface while maintaining complete data privacy and eliminating external API dependencies.

This combination is ideal for privacy-conscious individuals, developers building AI applications, organizations with sensitive data requirements, and anyone wanting to experiment with large language models without recurring costs. The stack provides enterprise-grade AI capabilities while keeping all data processing completely local, making it perfect for confidential document analysis, code generation in secure environments, and educational AI exploration without internet dependencies.

## Features

- Local LLM serving with OpenAI-compatible API for complete data privacy
- ChatGPT-like web interface with conversation history and user management
- Simple model management through Ollama's pull and run commands
- GPU acceleration support with NVIDIA CUDA for improved inference speed
- Multi-model support including Llama 2, Mistral, Code Llama, and custom Modelfiles
- RAG document support for context-aware conversations with uploaded files
- Prompt templates and custom personas for specialized AI interactions
- Model quantization options to optimize memory usage and performance

## Use cases

- Private AI chat for sensitive business communications and confidential document analysis
- Development teams building AI-powered applications without external API dependencies
- Educational institutions teaching AI concepts with hands-on local model experimentation
- Healthcare organizations requiring HIPAA-compliant AI assistance for medical documentation
- Legal firms using AI for document review while maintaining attorney-client privilege
- Home lab enthusiasts exploring large language models without subscription costs
- Offline AI applications in environments with limited or restricted internet access

## Prerequisites

- Minimum 8GB RAM (16GB+ recommended) for running 7B parameter models effectively
- NVIDIA GPU with 6GB+ VRAM for optimal performance, or remove GPU configuration for CPU-only mode
- Docker and Docker Compose installed with nvidia-container-toolkit for GPU support
- Ports 3000 and 11434 available for Open WebUI and Ollama API respectively
- Sufficient disk space for model storage (models range from 4GB to 70GB+)
- Basic understanding of LLM concepts and model parameter sizing

## docker-compose.yml

```yaml
services:
  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    restart: unless-stopped
    ports:
      - "${OLLAMA_PORT:-11434}:11434"
    volumes:
      - ollama_data:/root/.ollama
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities: [gpu]

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    restart: unless-stopped
    ports:
      - "${WEBUI_PORT:-3000}:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - open_webui_data:/app/backend/data
    depends_on:
      - ollama

volumes:
  ollama_data:
  open_webui_data:
```

## .env

```bash
# Ollama + Open WebUI
OLLAMA_PORT=11434
WEBUI_PORT=3000
```

## Usage notes

- Docs: https://docs.openwebui.com/, https://ollama.ai/docs
- Open WebUI at http://localhost:3000 - create admin on first visit
- Pull models: docker exec ollama ollama pull llama2:7b (or mistral, codellama)
- GPU requires nvidia-container-toolkit - remove deploy section for CPU mode
- Chat history, multiple users, and custom personas supported
- Connect additional OpenAI-compatible APIs in Settings > Connections

## Troubleshooting

- Open WebUI shows 'Ollama not found' error: Verify Ollama container is running and accessible at http://ollama:11434 internally
- Models fail to load with out of memory errors: Reduce model size or switch to CPU-only mode by removing the GPU deployment configuration
- GPU not detected in Ollama container: Install nvidia-container-toolkit on host system and restart Docker daemon
- Slow inference performance: Enable GPU acceleration or try smaller quantized models like 7B instead of 13B parameters
- Cannot access Open WebUI interface: Check if port 3000 is available and not blocked by firewall
- Model download fails or times out: Use docker exec ollama ollama pull command manually and check internet connectivity

