docker.recipes

Elasticsearch + Kibana

intermediate

Elasticsearch search engine with Kibana visualization dashboard.

Overview

Elasticsearch is a distributed, RESTful search and analytics engine built on Apache Lucene that has become the de facto standard for full-text search and log analytics. Originally developed by Shay Banon in 2010, Elasticsearch revolutionized how developers approach search functionality by providing near real-time indexing, powerful aggregations, and a simple REST API. As the core component of the Elastic Stack, it powers everything from application search features to enterprise-wide observability platforms. This stack combines Elasticsearch's search capabilities with Kibana, the visualization and management layer that transforms raw data into interactive dashboards, charts, and reports. Kibana acts as the window into your Elasticsearch data, providing both end-users with powerful search interfaces and administrators with cluster management tools. Together, they solve the challenge of making large volumes of structured and unstructured data searchable, analyzable, and visually comprehensible. This combination is ideal for development teams building search-driven applications, operations teams implementing centralized logging, and data analysts who need to explore and visualize large datasets. The stack excels in scenarios requiring real-time insights, complex aggregations, and intuitive data exploration interfaces, making it valuable for both technical and non-technical users who need to extract meaningful insights from their data.

Key Features

  • Full-text search with relevance scoring and advanced query DSL
  • Real-time data visualization with Kibana dashboards and charts
  • Distributed architecture with automatic sharding and replication
  • RESTful JSON API for programmatic data access and management
  • Powerful aggregations for analytics including histograms, terms, and metrics
  • Index lifecycle management for automatic data retention policies
  • Interactive data exploration with Kibana Discover and filtering
  • Machine learning capabilities for anomaly detection and forecasting

Common Use Cases

  • 1Application search functionality for e-commerce, documentation, or content sites
  • 2Centralized logging and log analysis for microservices architectures
  • 3Security information and event management (SIEM) for threat detection
  • 4Business intelligence dashboards for sales, marketing, and operational metrics
  • 5Application performance monitoring (APM) and observability platforms
  • 6Time-series data analysis for IoT sensors and monitoring systems
  • 7Content management systems requiring sophisticated search capabilities

Prerequisites

  • Minimum 4GB RAM available (2GB for Elasticsearch, additional for Kibana)
  • Docker Engine 20.10+ and Docker Compose v2 for optimal compatibility
  • Available ports 9200 (Elasticsearch API), 9300 (cluster communication), and 5601 (Kibana)
  • Basic understanding of REST APIs and JSON for data indexing and querying
  • Sufficient disk space for data storage and index growth over time

For development & testing. Review security settings, change default credentials, and test thoroughly before production use. See Terms

docker-compose.yml

docker-compose.yml
1services: 
2  elasticsearch: 
3    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
4    container_name: elasticsearch
5    restart: unless-stopped
6    environment: 
7      - discovery.type=single-node
8      - xpack.security.enabled=false
9      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
10    volumes: 
11      - es_data:/usr/share/elasticsearch/data
12    ports: 
13      - "9200:9200"
14      - "9300:9300"
15    networks: 
16      - elastic-network
17
18  kibana: 
19    image: docker.elastic.co/kibana/kibana:8.11.0
20    container_name: kibana
21    restart: unless-stopped
22    environment: 
23      ELASTICSEARCH_HOSTS: http://elasticsearch:9200
24    ports: 
25      - "5601:5601"
26    depends_on: 
27      - elasticsearch
28    networks: 
29      - elastic-network
30
31volumes: 
32  es_data: 
33
34networks: 
35  elastic-network: 
36    driver: bridge

.env Template

.env
1# Elasticsearch heap size (adjust based on available memory)
2ES_JAVA_OPTS=-Xms512m -Xmx512m

Usage Notes

  1. 1Docs: https://www.elastic.co/guide/en/elasticsearch/reference/current/index.html
  2. 2Access Kibana at http://localhost:5601 | ES API at http://localhost:9200
  3. 3Check cluster health: curl localhost:9200/_cluster/health?pretty
  4. 4Security disabled - enable xpack.security for production
  5. 5Adjust ES_JAVA_OPTS heap size (recommended: 50% of available RAM, max 32GB)
  6. 6Data stored in es_data volume - snapshot for backups

Individual Services(2 services)

Copy individual services to mix and match with your existing compose files.

elasticsearch
elasticsearch:
  image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
  container_name: elasticsearch
  restart: unless-stopped
  environment:
    - discovery.type=single-node
    - xpack.security.enabled=false
    - ES_JAVA_OPTS=-Xms512m -Xmx512m
  volumes:
    - es_data:/usr/share/elasticsearch/data
  ports:
    - "9200:9200"
    - "9300:9300"
  networks:
    - elastic-network
kibana
kibana:
  image: docker.elastic.co/kibana/kibana:8.11.0
  container_name: kibana
  restart: unless-stopped
  environment:
    ELASTICSEARCH_HOSTS: http://elasticsearch:9200
  ports:
    - "5601:5601"
  depends_on:
    - elasticsearch
  networks:
    - elastic-network

Quick Start

terminal
1# 1. Create the compose file
2cat > docker-compose.yml << 'EOF'
3services:
4  elasticsearch:
5    image: docker.elastic.co/elasticsearch/elasticsearch:8.11.0
6    container_name: elasticsearch
7    restart: unless-stopped
8    environment:
9      - discovery.type=single-node
10      - xpack.security.enabled=false
11      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
12    volumes:
13      - es_data:/usr/share/elasticsearch/data
14    ports:
15      - "9200:9200"
16      - "9300:9300"
17    networks:
18      - elastic-network
19
20  kibana:
21    image: docker.elastic.co/kibana/kibana:8.11.0
22    container_name: kibana
23    restart: unless-stopped
24    environment:
25      ELASTICSEARCH_HOSTS: http://elasticsearch:9200
26    ports:
27      - "5601:5601"
28    depends_on:
29      - elasticsearch
30    networks:
31      - elastic-network
32
33volumes:
34  es_data:
35
36networks:
37  elastic-network:
38    driver: bridge
39EOF
40
41# 2. Create the .env file
42cat > .env << 'EOF'
43# Elasticsearch heap size (adjust based on available memory)
44ES_JAVA_OPTS=-Xms512m -Xmx512m
45EOF
46
47# 3. Start the services
48docker compose up -d
49
50# 4. View logs
51docker compose logs -f

One-Liner

Run this command to download and set up the recipe in one step:

terminal
1curl -fsSL https://docker.recipes/api/recipes/elasticsearch-kibana/run | bash

Troubleshooting

  • Elasticsearch exits with 'max virtual memory areas vm.max_map_count too low': Run 'sysctl -w vm.max_map_count=262144' on host system
  • Kibana shows 'Kibana server is not ready yet': Wait 2-3 minutes for Elasticsearch to fully initialize before Kibana connects
  • OutOfMemoryError in Elasticsearch logs: Increase ES_JAVA_OPTS heap size or reduce it if exceeding available system RAM
  • Elasticsearch cluster status yellow or red: Check disk space availability and ensure proper index replica settings for single-node setup
  • Connection refused on port 9200: Verify Elasticsearch container started successfully and check Docker container logs for JVM errors

Community Notes

Loading...
Loading notes...

Download Recipe Kit

Get all files in a ready-to-deploy package

Includes docker-compose.yml, .env template, README, and license

Components

elasticsearchkibana

Tags

#elasticsearch#search#analytics#kibana

Category

Database Stacks

Related Recipes

Ad Space