Log all requests and responses with Django Middleware

From Django’s context:

Middleware is a framework of hooks into Django’s request/response processing.
It’s a light, low-level “plugin” system for globally altering Django’s input or output.

Middleware is a layer which has access to the request object before it is processed and response object after it is processed.

Instead of logging requests and responses in resp. views, its better to do at middleware layer which will log every incoming request. Why better?

this will log unhandled requests/views
one time job, no need to configure for every request/view
standardised logs for all requests/views

Here, we are using Django’s middleware semantics to construct a middleware which will log all requests and corresponding responses.

Create a middleware

Create middleware folder

Create a middleware folder in your project directory (directory in which settings.py exist).

1
2
mkdir middleware
touch middleware/__init__.py

middleware folder in same dir as settings.py

Create middleware file

Create a RequestLogMiddleware class in /middleware/request_log.py, see below:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

"""
Middleware to log `*/api/*` requests and responses.
"""
import socket
import time
import json
import logging

request_logger = logging.getLogger(__name__)

class RequestLogMiddleware:
    """Request Logging Middleware."""

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        start_time = time.monotonic()
        log_data = {
            "remote_address": request.META["REMOTE_ADDR"],
            "server_hostname": socket.gethostname(),
            "request_method": request.method,
            "request_path": request.get_full_path(),
        }

        # Only logging "*/api/*" patterns
        if "/api/" in str(request.get_full_path()):
            req_body = json.loads(request.body.decode("utf-8")) if request.body else {}
            log_data["request_body"] = req_body

        # request passes on to controller
        response = self.get_response(request)

        # add runtime to our log_data
        if response and response["content-type"] == "application/json":
            response_body = json.loads(response.content.decode("utf-8"))
            log_data["response_body"] = response_body
        log_data["run_time"] = time.time() - start_time

        request_logger.info(msg=log_data)

        return response

    # Log unhandled exceptions as well
    def process_exception(self, request, exception):
        try:
            raise exception
        except Exception as e:
            request_logger.exception("Unhandled Exception: " + str(e))
        return exception

Django allows us to write pre and post api-processing logic via __call__ method. Here we are logging only requests with /api/ in its path.

Activate the middleware

Activate the middleware by adding its path in MIDDLEWARE list in your Django’s application settings.py (below: last value in the middleware list, if you have other custom middlewares see ordering middleware)):

1
2
3
4
5
6
MIDDLEWARE = [
    ...
    "django.contrib.messages.middleware.MessageMiddleware",
    # Request Logger
    "<application>.middleware.request_log.RequestLogMiddleware",
]

Replace <application> with your application name.