How to use Grafana, Prometheus and Loki in Django for a better debug in your applications using Docker Compose

Nov 12, 2024

With the growing need to monitor and maintain application performance, tools like Grafana, Prometheus, and Loki have become essential for developers. These tools enable the collection, analysis, and visualization of metrics, as well as real-time log management.

In this article, we will explore how to integrate these tools with the Django framework, leveraging the power of Docker Compose to orchestrate the services. The goal is to create a complete environment where you can:

Monitor performance metrics with Prometheus.
Analyze structured logs with Loki.
Visualize data on dynamic dashboards in Grafana.

This integration will not only help with maintaining your application but also serve as a practical introduction to observability tools within the Django ecosystem. Let's get started!

1. Install django-prometheus in your application

Requirements

Django > 3.2
Python 3.7 and above.

Installation

Install with:

Terminal

pip install django-prometheus

Quickstart

In your settings.py:

./settings.py

INSTALLED_APPS = [   ...   'django_prometheus',   ...] MIDDLEWARE = [    'django_prometheus.middleware.PrometheusBeforeMiddleware',    # All your other middlewares go here, including the default    # middlewares like SessionMiddleware, CommonMiddleware,    # CsrfViewmiddleware, SecurityMiddleware, etc.    'django_prometheus.middleware.PrometheusAfterMiddleware',]

In your urls.py:

./urls.py

urlpatterns = [   ...    path('', include('django_prometheus.urls')),]

Monitoring your databases

SQLite, MySQL, and PostgreSQL databases can be monitored. Just replace the ENGINE property of your database, replacing django.db.backends with django_prometheus.db.backends.

./settings.py

DATABASES = {    'default': {        'ENGINE': 'django_prometheus.db.backends.sqlite3',        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),    },}

Monitoring your caches

Filebased, memcached, redis caches can be monitored. Just replace the cache backend to use the one provided by django_prometheusdjango.core.cache.backends with django_prometheus.cache.backends.

./settings.py

CACHES = {    'default': {        'BACKEND': 'django_prometheus.cache.backends.filebased.FileBasedCache',        'LOCATION': '/var/tmp/django_cache',    },}

Monitoring your models

You may want to monitor the creation/deletion/update rate for your model. This can be done by adding a mixin to them. This is safe to do on existing models (it does not require a migration).

If your model is:

./models.py

class Dog(models.Model):    name = models.CharField(max_length=100, unique=True)    breed = models.CharField(max_length=100, blank=True, null=True)    age = models.PositiveIntegerField(blank=True, null=True)

Just add the ExportModelOperationsMixin as such:

./models.py

from django_prometheus.models import ExportModelOperationsMixin class Dog(ExportModelOperationsMixin('dog'), models.Model):    name = models.CharField(max_length=100, unique=True)    breed = models.CharField(max_length=100, blank=True, null=True)    age = models.PositiveIntegerField(blank=True, null=True)

Used as reference: https://pypi.org/project/django-prometheus/

1. Setup up docker-compose in your application

Logging

Grafana Loki
Grafana Promtail

Installation

Loki is a log aggregation and indexing system, developed by Grafana Labs, optimized to function as a database for logs and the Promtail is the log collection agent that sends logs to Loki. It acts as a log shipper.

./docker-compose-dev.yml

  loki:    image: grafana/loki:2.9.2    ports:      - "3100:3100"    command: -config.file=/etc/loki/local-config.yaml    networks:      - loki   promtail:    image: grafana/promtail:2.9.2    volumes:      - /var/log:/var/log      - /var/run/docker.sock:/var/run/docker.sock      - ./promtail/config/promtail-config.yml:/etc/promtail/config.yml    command: -config.file=/etc/promtail/config.yml    networks:      - loki

In the Promtail job we can see that we are mounting the ./promtail/config/promtail-config.yml that is the file who will be responsible configurate Promtail and send the logs to Loki. So let's create this file.

./promtail/config/promtail-config.yml

server:  http_listen_port: 9080  grpc_listen_port: 0 positions:  filename: /tmp/positions.yaml clients:  - url: http://loki:3100/loki/api/v1/push scrape_configs:  - job_name: system    static_configs:      - targets:          - localhost        labels:          job: varlogs          __path__: /var/log/*log   - job_name: docker    docker_sd_configs:      - host: unix:///var/run/docker.sock    relabel_configs:      - source_labels: [__meta_docker_container_name]        target_label: container      - source_labels:          [__meta_docker_container_label_com_docker_compose_service]        target_label: service      - source_labels:          [__meta_docker_container_label_com_docker_compose_project]        target_label: compose_project      - source_labels: [__meta_docker_container_label_com_docker_compose_oneoff]        target_label: compose_oneoff      - source_labels:          [__meta_docker_container_label_com_docker_compose_config_hash]        target_label: compose_config_hash      - source_labels:          [__meta_docker_container_label_com_docker_compose_version]        target_label: compose_version

Metrics and Dashboard

Prometheus
Grafana

Installation

Grafana is a tool for creating and visualizing dashboards based on data from various sources and Prometheus is a monitoring tool and alert system focused on numerical metrics based on time series.

./docker-compose-dev.yml

grafana:  environment:    - GF_PATHS_PROVISIONING=/etc/grafana/provisioning  entrypoint:    - sh    - -euc    - |      mkdir -p /etc/grafana/provisioning/datasources      cat <<EOF > /etc/grafana/provisioning/datasources/ds.yaml      apiVersion: 1      datasources:      - name: Loki        type: loki        access: proxy        orgId: 1        url: http://loki:3100        basicAuth: false        isDefault: true        version: 1        editable: false      - name: Prometheus        type: prometheus        access: proxy        orgId: 1        url: http://prometheus:9090        basicAuth: false        isDefault: false        version: 1        editable: false      EOF      /run.sh  image: grafana/grafana:latest  volumes:    - grafana_data:/var/lib/grafana  ports:    - "3060:3000"  networks:    - loki prometheus:  image: prom/prometheus:latest  ports:    - "9090:9090"  entrypoint:    - sh    - -c    - |      /bin/prometheus --config.file=/etc/prometheus/prometheus/config/prometheus.yml --web.enable-lifecycle  volumes:    - ./prometheus/config/prometheus.yml:/etc/prometheus/prometheus/config/prometheus.yml    - prometheus_data:/prometheus  networks:    - loki

In the Prometheus job we can see that we are mounting the ./prometheus/config/prometheus.yml that is the file who will be responsible configurate Prometheus and get the metrics from Django application. So let's create this file.

./prometheus/config/prometheus.yml

global:  scrape_interval: 15s  scrape_timeout: 10s  scrape_protocols:    - OpenMetricsText1.0.0    - OpenMetricsText0.0.1    - PrometheusText0.0.4  evaluation_interval: 15sruntime:  gogc: 75scrape_configs:  - job_name: prometheus    honor_timestamps: true    track_timestamps_staleness: false    scrape_interval: 15s    scrape_timeout: 10s    scrape_protocols:      - OpenMetricsText1.0.0      - OpenMetricsText0.0.1      - PrometheusText0.0.4    metrics_path: /metrics    scheme: http    enable_compression: true    follow_redirects: true    enable_http2: true    static_configs:      - targets:          - <YOUR IP>:8000

Detail: Replace <YOUR IP> with your IP address in dev ambient.

Final touches

Volumes
Network

./docker-compose-dev.yml

networks:  loki: volumes:  grafana_data:  prometheus_data:

Closing Thoughts

Integrating Grafana, Prometheus, and Loki into a Django project using Docker Compose creates a powerful environment for monitoring, metric analysis, and log management. This setup provides a comprehensive view of your application's behavior, making it easier to identify issues and optimize performance.

Throughout this article, we explored how each tool contributes to the observability ecosystem:

Prometheus collects essential metrics.
Loki efficiently stores and organizes logs.
Grafana unifies the data into interactive and dynamic dashboards.

With this stack configured, you're ready to monitor your application professionally, anticipating issues and ensuring a more stable experience for your users. Now it's time to customize and expand this environment to fit your specific needs.

If you found this guide helpful, feel free to share it and if you have any questions or comments just contact me. Your journey to mastering monitoring and observability has just begun! 🚀