Middleware In Django

Introduction to Middleware

TL;DR - Middleware is a piece of software or a program that acts as a communication bridge between other sets of software or programs.

Middleware is an application/set of applications that allows an application/set of applications to communicate with another application/set of applications.

Confused? Okay, try this one.

A Chinese and an English man are trying to have a conversation. They do not understand each other. As you know both, Chinese and English, you listen to the Chinese man and translate to the English man. Here, you are acting as a middleman and your role is a translator!

As you, as a human, can act as a middleman to ease some set of operations, we can also use some software between other sets of software or sets of programs to make the flow easier.

Hence, middleware is a piece of software that can essentially interact with another software or server to create a communication bridge between them so that business transactions can be made easier.

Why do we need middleware, did you ask? Picture this. You have a very old Walkman, and you need to connect it to your computer. How would you connect? You installed a software that allowed you to connect to the Walkman. What's the software you installed? It's a driver for that Walkman! The driver is making a connection between your computer and the Walkman so that they can connect with ease and enable easy communications.

Middleware does not need to be a fully-equipped software that connects two or more other software, it might be a simple program that may connect other programs in a computer system. Here middleware in web framework comes into play.

All the web frameworks use middleware to segregate the essential tasks before proceeding to execute the business logic of the platform. Assume this - you have written all the code that received an HTTP request, decided on the CSRF token, validated auth token, checked for any exception, and written your business logic or APIs in a single place. Does it seem interesting? No. So, all the web frameworks segregate these essential codes and apply these codes to the overall framework. As these codes a separated any requests are first handled in these codes, and then passed to the actual views/APIs you write.

These separate codes act as middlemen in the web framework - and are referred to as middleware! We will dive in about middleware in Django, but remember, irrespective of web frameworks, middleware in web frameworks works similarly.

Middleware Visualize

TL;DR - Middleware in Django is only active during the request/response cycle and each middleware is responsible for a specific task.

Middleware is a framework of hooks into Django’s request/response processing. It’s a light, low-level “plugin” system for globally altering Django’s input or output.

Django Documentation

Essentially, as we have discussed, the documentation quotes the middleware in Django as a framework of hooks and these hooks are applied only during the request and response cycle. What does hook mean, did you just ask? The name refers to its meaning. Hooks are something that has characteristics of adding/removing or switching on/switching off to something. If you add a hook named X, to A, the A will have the functionalities A + X. If you remove X from A, A will have functionalities only A.

Each middleware in Django has its respective task to perform. For example, AuthenticationMiddleware is responsible for validating the authentication of a user. We will talk about these later.

There are two phases when middleware gets activated - at the HTTP request time when a client makes an HTTP request. The Django middlewares inspect the request on a few levels, just like Onion - this is the request phase. The other phase is when the middlewares are back into action when the view is executed and a response is returned - it's called the response phase. Again, the response goes through a few levels of inspections by various middleware, and then it is flushed into the internet to travel to the client.

Still confused? Picture this - You want to visit a gangster for some reason. The gangster is on the top floor of the building. You start from the first floor, and on each floor, the security guards are checking you for any threat to the gangster. If they find you safe for the gangster, then you will be allowed to meet him. Once you meet him, and complete the meeting, you start from the top floor to return. Again, the security personnel on each floor check you for anything expensive you have stolen from any floor. If only they find you clean, you will be allowed to exit the building.

Remember this analogy because this will help you to understand how middleware works internally.

Middleware Execution Sequence in Django

How To Activate/ Deactivate Middleware in Django

TL;DR - Add the full Django path of your custom middleware in the MIDDLEWARE list in the settings.py file to activate and remove to deactivate your custom middleware.

There are two types of middleware in Django.

built-in middleware
custom middleware

Built-in Middleware in Django

MIDDLEWARE = [
    "django.middleware.security.SecurityMiddleware",
    "django.contrib.sessions.middleware.SessionMiddleware",
    "django.middleware.common.CommonMiddleware",
    "django.middleware.csrf.CsrfViewMiddleware",
    "django.contrib.auth.middleware.AuthenticationMiddleware",
    "django.contrib.messages.middleware.MessageMiddleware",
    "django.middleware.clickjacking.XFrameOptionsMiddleware",
]

You can see the built-in middleware in the settings.py file in the MIDDLEWARE list. These middlewares are automatically added when you run the django-admin startproject command to create a project in Django.

Once you create your custom middleware, you just need to add the full Django path at the end of the MIDDLEWARE list. This will allow the middleware to be applied all over your Django project. In case you no longer need to use your custom middleware, just remove the full Django path from the MIDDLEWARE list. This will ensure your middleware will not be applied to your Django project anymore. This specific functionality is called "plug-in" or "unplug".

Types of Middleware

TL;DR - In Django, middleware can be written as a closure of Python (function-based) or class-based. All the built-in middlewares in Django are class-based.

The ordering in the MIDDLEWARE is very important as a middleware's functionality might be dependent on other middleware. Hence, the independent middleware must be executed before the dependent middleware. Initially django stored the middleware in tuples, but now it stores them in a list.

Function-Based Middleware

def simple_middleware(get_response):
    # One-time configuration and initialization.

    def middleware(request):
        # Code to be executed for each request before
        # the view (and later middleware) are called.

        response = get_response(request)

        # Code to be executed for each request/response after
        # the view is called.

        return response

    return middleware

Class-Based Middleware

class SimpleMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        # Sequence 01: 
        # One-time configuration and initialization when the server starts 

    def __call__(self, request):
        # Sequence 02: Code to be executed for each request before
        # the view (and later middleware) are called.

        # Sequence 03: Request phase hooks are executed (if any) 

        # Sequnce 04: 
        # Recursive call to next middleware or the actual view if 
        # this is the last middleware 
        response = self.get_response(request)

        # Sequence 05: Code to be executed in Response phase after
        # the view is called.

        # Sequence 06: Response phase hooks are executed before Sequence 05. (if any)


        return response  # activates the list[i-1] middleware.

Hooks in Middleware

TL;DR - Some plug-ins are available to add in class-based middleware in Django to enable low-level access to the request/response cycle.

Middleware hooks are plug-ins of plug-ins! If middleware in Django is a plug-in to a Django project that we can add or remove in the MIDDLEWARE list, the hooks in middleware are plug-ins for that middleware. These hooks can be added or removed to enhance the functionalities of the middleware. These additional hooks for middleware are provided by Django to empower the developer to manipulate the request and response at a low level.

Request Phase Hooks

The request phase hooks are applied in the request phase i.e. before the actual view is executed. These hooks are applied in normal order - from Up to Bottom order of the MIDDLEWARE list. At this phase, there are two hooks available.
- process_request() -
  
  This hook is called on each request before Django decides which view to call using the ROOT_URLCONF. This hook returns None, to proceed with the request to other middleware.
  
  If HttpResponse is returned, Django will directly apply response middleware and return a response. No other code will be executed. This way, using a hook, we can inspect and monitor if we want to allow a request to enter into the system.
- process_view()
  
  process_view is called just before the actual view is called. It can return either None or HttpResponse. If None is returned, other middleware will be executed. If HttpResponse is returned (The hook can create and return custom HttpResponse), Django will directly apply response middleware and return a response. No other code will be executed.
Response Phase Hooks

Response phase hooks are applied after the actual view is executed. These hooks are applied in reverse order, from Bottom to Up order of the MIDDLEWARE list. In this phase, there are three hooks available.
- process_exception()
  
  This hook is executed only if an exception occurs during the execution of the view. This hook either returns None, if an exception has occurred or returns a HttpResponse if no exception has occurred.
- process_template_response()
  
  This hook is only applied if the response it received is an Django.template.response.TemplateResponse object. If the response has a render() method, it is also applied. When we use TemplateResponse() or render() to render a html file, in such cases, this hook is applied.
  
  We can also alter the template to render, the context data within this hook. process_template_response hook, if applied, must return the TemplateResponse object it received, or the hook can create an TemplateResponse object and return it.
- process_response()
  
  process_response is the only hook in Django middleware that is always executed - no matter if the other hooks stopped the request and returned a custom response, or the view is successfully executed and returned a response. process_response must return an HttpRequest response, either received from the other middleware or create and return a custom HttpResponse.

Middleware in Django Works Like Layer of Onion

How Middleware Works in Django - Anatomy of A Middleware

Any given middleware generally has six stages of processes.

Sequence 01

The initial or activation of the middleware when the server starts. The __init__ is called at this time.

 class SimpleMiddleware:
     def __init__(self, get_response):
         self.get_response = get_response
         # Sequence 01

Sequence 02

The __call__ method is used to make any Python instance callable, and for each incoming HTTP request, the __call__ is triggered. The subsequent code, under __call__ and before the get_response(), is executed at this time.
```
 def __call__(self, request):
     # Sequence 02
```
Sequence 03

After the subsequent code of __call__ has been executed, if any request phase hooks are available in the middleware, then at this time, the request phase hooks are called.
```
 def __call__(self, request):
     # After Sequence 02
     # Sequence 03 (Request phase hooks executed after sequence 02)
```
Sequence 04

The recursive call to the next middleware or the actual view, if this is the last middleware in the layer - response = get_response(request)

The current middleware does not know whether they would receive the response from the actual view, or any subsequent middleware. They just call the next middleware unaware of whether the next thing is middleware or a Django view.
```
 # Sequence 04 - Recursivce call to get the response 
 response = self.get_response(request)
```
Sequence 05

At sequence 04, the response variable will receive some value after the actual view has returned an HttpResponse or TemplateResponse, or any subsequent middleware has returned a HttpResponse or None; and the code below to this recursive call will be executed.
```
 response = self.get_response(request)
 # Sequence 05 - As response has received value, the recursive calls ended. 
 # And below code of this response variable will be executed. 

 return response # activates the list[i-1] middleware.
```
Sequence 06

If the middleware has any response phase hook, then the response phase hook is applied first, then any code below response = get_response(request) is applied.
```
 response = self.get_response(request)

 # If the middleware has response phase hooks defined, then these hooks are 
 # executed before Sequence 05. In such cases, Sequence 06 is applied, then
 # Sequence 05 is executed. 

 return response # activates the list[i-1] middleware.
```
What's Happening Inside - The Backstage
```
 response = self.get_response(request)
```
This line is a recursive call to other middleware. As all the middleware has this basic setup, get_response(request) will eventually either call the actual view or get an early response from other middleware.

As calling this line gets us to the actual view, we can consider this is why the code upper to this line is called at the request phase for each middleware.

When the view has returned either HttpResponse or TemplateResponse object, the response variable has received value from the recursive call, this ends the recursive calls. This resonates with us why the below code of the response variable is executed at the response phase.

These recursive calls are happening at the cost of max len of MIDDLEWARE list as either the request would reach the actual view, then all the middleware would call response = get_response(request), or at any given position, any middleware could return a response.

Why Response Phase Middleware Run in Reverse Way

As you have probably read in the Hooks In Middleware section, I have mentioned that request phase hooks run from UP to bottom, and response phase middleware runs from Bottom to UP fashion in MIDDLEWARE list.

When I talk about from UP to Bottom, it means, the hooks/middlewares are being applied from the 0th index to the Nth index of the list. At the response time, Bottom-UP means the hooks/middlewares are being applied from the Nth to 0th index. You have probably guessed why at the request phase the execution is happening from the Nth to 0th index reading the previous block.

Did not get it? let's go again.

Do you remember, I gave an analogy of gangster? It applies the same here.

As the view has expensive business logic like the gangster, we will not allow malicious requests to reach the view. All the floors are the middleware - the checkings are respective tasks the middlewares are assigned to do.

As you have reached the top floor after various checks - the request phase ended. You start from the top floor to come to the ground floor to exit the building - in the response phase - the middlewares are applied in reverse order. When you finish meeting with the gangster, you are again checked on each floor if you stole anything precious - the middleware again checks the response if complies with the protocol.

Wrap Up

Middlewares in Django handle various security and auth-related tasks and decide whether to allow or deny a request or response - it makes developers' lives easier by making the platform safe.
Middleware in Django also provides support for async-type middleware.
Middleware in Django supports extensive exception handling.
The initial execution of the middleware ( the code inside the __init__ ) applied from the Nth index to the 0th index when the server starts.
In the request phase, the middlewares are executed from the 0th index to the Nth index of the MIDDLEWARE list in settings.py
In the response phase, the middlewares are executed from the Nth index to the 0th index due to recursive calls.

Conclusion

This was it, folks, about the Middleware in Django. If you have any questions regarding Middleware in Django or anything related to backend development, feel free to comment.

Wait, here's a question for you: Do you know the differences between a Cloud Engineer and a DevOps Engineer? Well, here's a one-liner - Cloud engineers primarily focus on the design and management of cloud infrastructure, while DevOps engineers concentrate on streamlining development and operations through automation and collaboration.

And here's some general health advice – use an ergonomic chair dev, and take care of your health. After all, health is wealth.

Understanding Middleware in Django - A Complete Handbook

Understanding How Middleware Works - A Complete Guide

Table of contents

Introduction to Middleware