Centralized Logging for DevOps: ELK Stack vs Grafana Loki

Your application is spread across 20 containers. When something breaks at 3 AM, you need to know what happened, where, and when — without SSHing into every host and grepping log files. That's what centralized logging solves. The two dominant open-source options are the ELK Stack (Elasticsearch, Logstash, Kibana) and Grafana Loki. They look similar on the surface, but they're built on completely different philosophies. This guide breaks down the trade-offs and shows you working setups for both.

The Core Difference: Index Everything vs Label Everything

ELK indexes the full content of every log line. Elasticsearch parses, tokenizes, and stores each field, which makes full-text search instant — but storage is expensive. Loki takes the opposite approach: it indexes only labels (like app, namespace, pod) and stores the raw log content as compressed blocks. Queries still scan the data, but with chunk compression and object storage backing, Loki is dramatically cheaper at scale.

Rule of thumb: ELK if you need deep field-level analytics and alerting on structured data. Loki if you mostly need "grep across all my servers, fast and cheap."

ELK Stack Setup with Docker Compose

A minimal ELK for development can run in a single Compose file. Note the ES_JAVA_OPTS — Elasticsearch is a JVM app and will eat your RAM if you let it.

services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch:8.14.0
    environment:
      - discovery.type=single-node
      - xpack.security.enabled=false
      - ES_JAVA_OPTS=-Xms512m -Xmx512m
    ports: ["9200:9200"]
    volumes: [esdata:/usr/share/elasticsearch/data]
  logstash:
    image: docker.elastic.co/logstash/logstash:8.14.0
    depends_on: [elasticsearch]
    volumes:
      - ./logstash.conf:/usr/share/logstash/pipeline/logstash.conf
  kibana:
    image: docker.elastic.co/kibana/kibana:8.14.0
    depends_on: [elasticsearch]
    ports: ["5601:5601"]
volumes:
  esdata:

Logstash's job is to parse incoming logs (from Filebeat, syslog, or HTTP) and transform them before indexing. A classic pipeline:

input { beats { port => 5044 } }
filter {
  grok {
    match => { "message" => "%{TIMESTAMP_ISO8601:ts} %{LOGLEVEL:level} %{GREEDYDATA:msg}" }
  }
  date { match => [ "ts", "ISO8601" ] }
}
output { elasticsearch { hosts => ["elasticsearch:9200"] } }

You query it in Kibana with KQL — for example level: "ERROR" and service: "payment" — and get results in milliseconds because every field is indexed.

Loki Setup with Docker Compose

Loki is simpler to run because it has no Java and no separate shipper requirement — Promtail scrapes Docker logs directly. A minimal single-binary Loki (v3 mode) plus Promtail and Grafana:

services:
  loki:
    image: grafana/loki:3.2.0
    command: -config.file=/etc/loki/local-config.yaml
    ports: ["3100:3100"]
    volumes: [loki-data:/loki]
  promtail:
    image: grafana/promtail:3.2.0
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - ./promtail.yaml:/etc/promtail/promtail.yaml
  grafana:
    image: grafana/grafana:11.1.0
    ports: ["3000:3000"]
    environment:
      - GF_AUTH_ANONYMOUS_ENABLED=true
      - GF_SECURITY_ADMIN_PASSWORD=admin
volumes:
  loki-data:

Promtail auto-discovers containers and attaches Docker labels as Loki labels:

scrape_configs:
  - job_name: docker
    docker_sd_configs:
      - host: unix:///var/run/docker.sock
    relabel_configs:
      - source_labels: [__meta_docker_container_label_com_docker_compose_service]
        target_label: service
      - source_labels: [__meta_docker_container_name]
        target_label: container

Queries use LogQL. Select logs by labels, then filter with a stream selector and line expressions:

{service="payment"} |= "ERROR" | json | level="error"

LogQL v2 adds powerful aggregation — counting errors per service over time:

sum by (service) (count_over_time({level="error"}[5m]))

Head-to-Head Comparison

Production Best Practices

• Start with Loki for Kubernetes. Loki was designed around Kubernetes label semantics (namespace, pod, container). The Promtail/K8s integration is mature; ELK on K8s requires more tuning.
• Pin retention with a storage budget. Loki: set retention_period in config. ELK: use Index Lifecycle Management (ILM) to roll and delete old indices automatically.
• Never ship secrets. Log redaction matters — filter passwords, tokens, and PII at the shipper (Logstash filter / Promtail pipeline stage) before they hit storage.
• Structure your logs. JSON logs are dramatically easier to query in both systems than multiline text. Use a JSON formatter in your app.
• Watch cardinality in Loki. A label like request_id with millions of unique values will destroy performance. Keep labels to tens of distinct values.
• Alert on top of both. ELK: Watcher or ElastAlert. Loki: Grafana alerting with LogQL metric queries — no extra component needed.

Which One Should You Choose?

For a small team running Docker Compose workloads, Loki + Grafana is the pragmatic default: one binary to run, cheap storage, and you probably already use Grafana for metrics. Reach for ELK when you need full-text search over structured fields, security analytics (SIEM), or deep Kibana visualizations — and you have the ops capacity to run and tune Elasticsearch in production.

Both stacks are legitimate. The mistake is choosing one without considering your storage budget and query patterns. Logs grow forever — pick the architecture you can afford to keep.