docker.recipes

Huginn + MySQL

intermediate

Create agents that monitor and act on your behalf.

Overview

Huginn is an open-source automation platform designed to create intelligent agents that monitor websites, APIs, and data sources while executing actions based on predefined rules. Originally inspired by Yahoo Pipes and IFTTT, Huginn allows users to build complex workflows that can scrape websites, process RSS feeds, send notifications, and integrate with hundreds of web services through a visual agent-based interface. The platform excels at creating sophisticated automation chains where the output of one agent becomes the input for another, enabling powerful data processing and monitoring capabilities. This stack combines Huginn's Ruby on Rails application with MySQL as the backend database, providing a robust foundation for storing agent configurations, collected data, and event logs. MySQL's proven reliability and performance characteristics make it ideal for handling Huginn's frequent database operations, including agent state management, event storage, and user data persistence. The MySQL integration ensures that complex agent workflows and their historical data remain consistently available and performant. This combination is perfect for power users, system administrators, and automation enthusiasts who need a self-hosted alternative to commercial automation services like Zapier or Microsoft Flow. Organizations concerned with data privacy, those requiring custom automation logic, or users who need to monitor internal systems will find this stack particularly valuable for creating comprehensive monitoring and automation solutions without relying on third-party services.

Key Features

  • Visual agent workflow builder with 40+ pre-built agent types including Website, Email, Twitter, and Webhook agents
  • Advanced web scraping capabilities with CSS selector and XPath support for extracting structured data
  • Event-driven architecture where agents can trigger other agents based on conditions and schedules
  • Built-in credential management system for securely storing API keys and authentication tokens
  • Liquid templating engine for dynamic data transformation and conditional logic processing
  • MySQL-backed event logging with full audit trail and historical data retention
  • RESTful API for programmatic agent management and external system integration
  • Dry-run mode for testing agent configurations before deployment

Common Use Cases

  • 1Website monitoring and change detection for competitor pricing, product availability, or content updates
  • 2Social media monitoring and automated response systems for brand mentions and customer service
  • 3IT infrastructure monitoring with custom alerting rules and escalation procedures
  • 4RSS feed aggregation and content curation with filtering and redistribution capabilities
  • 5E-commerce inventory tracking and price monitoring across multiple platforms
  • 6Weather data collection and automated notifications for specific conditions
  • 7API endpoint monitoring with custom validation rules and failure notifications

Prerequisites

  • Docker and Docker Compose installed with at least 2GB available RAM for MySQL operations
  • Port 3000 available for Huginn web interface access
  • Basic understanding of web scraping concepts, CSS selectors, and HTTP requests
  • Familiarity with JSON data structures and Liquid template syntax for agent configuration
  • Environment variables configured for database credentials, admin user, and invitation codes
  • Network connectivity for external API calls and website monitoring targets

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  huginn: 
3    image: ghcr.io/huginn/huginn:latest
4    environment: 
5      - DOMAIN=localhost
6      - PORT=3000
7      - DATABASE_ADAPTER=mysql2
8      - DATABASE_HOST=mysql
9      - DATABASE_NAME=huginn
10      - DATABASE_USERNAME=${MYSQL_USER}
11      - DATABASE_PASSWORD=${MYSQL_PASSWORD}
12      - RAILS_ENV=production
13      - FORCE_SSL=false
14      - INVITATION_CODE=${INVITATION_CODE}
15      - SEED_USERNAME=${ADMIN_USER}
16      - SEED_PASSWORD=${ADMIN_PASSWORD}
17    volumes: 
18      - huginn-data:/var/lib/huginn
19    ports: 
20      - "3000:3000"
21    depends_on: 
22      - mysql
23    networks: 
24      - huginn-network
25    restart: unless-stopped
26
27  mysql: 
28    image: mysql:8.0
29    environment: 
30      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
31      - MYSQL_DATABASE=huginn
32      - MYSQL_USER=${MYSQL_USER}
33      - MYSQL_PASSWORD=${MYSQL_PASSWORD}
34    volumes: 
35      - mysql-data:/var/lib/mysql
36    networks: 
37      - huginn-network
38    restart: unless-stopped
39
40volumes: 
41  huginn-data: 
42  mysql-data: 
43
44networks: 
45  huginn-network: 
46    driver: bridge

.env Template

.env
1# Huginn
2MYSQL_ROOT_PASSWORD=secure_root_password
3MYSQL_USER=huginn
4MYSQL_PASSWORD=secure_mysql_password
5ADMIN_USER=admin
6ADMIN_PASSWORD=secure_admin_password
7INVITATION_CODE=your_invitation_code

Usage Notes

  1. 1Web UI at http://localhost:3000
  2. 2Create agents for tasks
  3. 3Website monitoring
  4. 4Event processing
  5. 5Many pre-built agent types

Individual Services(2 services)

Copy individual services to mix and match with your existing compose files.

huginn
huginn:
  image: ghcr.io/huginn/huginn:latest
  environment:
    - DOMAIN=localhost
    - PORT=3000
    - DATABASE_ADAPTER=mysql2
    - DATABASE_HOST=mysql
    - DATABASE_NAME=huginn
    - DATABASE_USERNAME=${MYSQL_USER}
    - DATABASE_PASSWORD=${MYSQL_PASSWORD}
    - RAILS_ENV=production
    - FORCE_SSL=false
    - INVITATION_CODE=${INVITATION_CODE}
    - SEED_USERNAME=${ADMIN_USER}
    - SEED_PASSWORD=${ADMIN_PASSWORD}
  volumes:
    - huginn-data:/var/lib/huginn
  ports:
    - "3000:3000"
  depends_on:
    - mysql
  networks:
    - huginn-network
  restart: unless-stopped
mysql
mysql:
  image: mysql:8.0
  environment:
    - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
    - MYSQL_DATABASE=huginn
    - MYSQL_USER=${MYSQL_USER}
    - MYSQL_PASSWORD=${MYSQL_PASSWORD}
  volumes:
    - mysql-data:/var/lib/mysql
  networks:
    - huginn-network
  restart: unless-stopped

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  huginn:
5    image: ghcr.io/huginn/huginn:latest
6    environment:
7      - DOMAIN=localhost
8      - PORT=3000
9      - DATABASE_ADAPTER=mysql2
10      - DATABASE_HOST=mysql
11      - DATABASE_NAME=huginn
12      - DATABASE_USERNAME=${MYSQL_USER}
13      - DATABASE_PASSWORD=${MYSQL_PASSWORD}
14      - RAILS_ENV=production
15      - FORCE_SSL=false
16      - INVITATION_CODE=${INVITATION_CODE}
17      - SEED_USERNAME=${ADMIN_USER}
18      - SEED_PASSWORD=${ADMIN_PASSWORD}
19    volumes:
20      - huginn-data:/var/lib/huginn
21    ports:
22      - "3000:3000"
23    depends_on:
24      - mysql
25    networks:
26      - huginn-network
27    restart: unless-stopped
28
29  mysql:
30    image: mysql:8.0
31    environment:
32      - MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
33      - MYSQL_DATABASE=huginn
34      - MYSQL_USER=${MYSQL_USER}
35      - MYSQL_PASSWORD=${MYSQL_PASSWORD}
36    volumes:
37      - mysql-data:/var/lib/mysql
38    networks:
39      - huginn-network
40    restart: unless-stopped
41
42volumes:
43  huginn-data:
44  mysql-data:
45
46networks:
47  huginn-network:
48    driver: bridge
49EOF
50
51# 2. Create the .env file
52cat > .env << 'EOF'
53# Huginn
54MYSQL_ROOT_PASSWORD=secure_root_password
55MYSQL_USER=huginn
56MYSQL_PASSWORD=secure_mysql_password
57ADMIN_USER=admin
58ADMIN_PASSWORD=secure_admin_password
59INVITATION_CODE=your_invitation_code
60EOF
61
62# 3. Start the services
63docker compose up -d
64
65# 4. View logs
66docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/huginn-automation/run | bash

Troubleshooting

  • Huginn container fails with 'database connection refused': Ensure MySQL container is fully initialized before Huginn starts by adding a health check or increasing startup delay
  • Agent execution fails with 'memory limit exceeded': Increase Docker container memory allocation as complex web scraping operations require additional RAM
  • Web interface shows 'Invalid invitation code': Set the INVITATION_CODE environment variable and use it when creating new user accounts
  • MySQL container restarts continuously: Check that MYSQL_ROOT_PASSWORD is properly set and contains no special characters that might cause parsing issues
  • Agent schedules not executing: Verify that the Rails background job processor is running and check Huginn logs for delayed_job worker status
  • Website Agent returns empty results: Inspect the target website's structure and update CSS selectors, as many sites implement anti-scraping measures or change their HTML structure

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

huginnmysql

Tags

#huginn#automation#agents#monitoring#web-scraping

Category

Productivity & Collaboration

Related Recipes

Ad Space