Details of Django Middleware

Time:2020-6-2

Django advanced practical programming

Video tutorial sharing address:

https://study.163.com/course/introduction/1209407824.htm?share=2&shareId=400000000535031

Use the link address above to enter

Details of Django Middleware

Intron

Middleware is a hook framework used by Django to handle requests and responses. It is a lightweight, low-level “plug-in” system, used to control the input or output of Django globally, which can be understood as a built-in app or small framework.

staydjango.core.handlers.baseThe module defines how to access middleware, which is also one of the entrances to learn Django source code.

Each middleware component is responsible for some specific functions. For example, Django includes a middleware componentAuthenticationMiddleware, which uses session mechanism to connect users with requestsrequestConnect.

Middleware can be placed anywhere in your project and accessed as a python path.

Django has some built-in middleware, and some of them are opened automatically. We can adjust it according to our own needs.

How to enable Middleware

To enable middleware components, add them to the Django configuration filesettings.pyOfMIDDLEWAREConfiguration item list.

stayMIDDLEWAREMiddleware is represented by strings. This string, separated by dots, points to the full Python path to the class or function name of the middleware factory. Here’s how to usedjango-admin startprojectAfter the command creates a project, the default middleware configuration is:

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

In fact, you can use no Middleware in Django, if you like,MIDDLEWAREConfiguration item can be empty. However, it is strongly recommended to use at leastCommonMiddleware。 My suggestion is to keep the default configuration, which will help you improve the security of the website.

The most critical sequence problem of Middleware

MIDDLEWAREThe order is very important, because some middleware will depend on other middleware. For example:AuthenticationMiddlewareThe authenticated user information stored in the session middleware is required, so it must be stored in theSessionMiddlewareRun after.

In the request phase, Django executes the middleware in the defined order before invoking the viewMIDDLEWARE, top down.

You can think of it as an onion: each middleware class is a “layer” that wraps the core of the onion – the actual business view. If the request passes through all the middleware layers of the onion to the view of the kernel, the response will pass through each middleware layer in reverse order during the return process, and finally return to the user.

If the execution process of a layer thinks that the current request should be rejected, or some errors occur, resulting in a short circuit, and a response is returned directly, then the remaining middleware and core view functions will not be executed.

Django’s built-in Middleware

Django has built in the following middleware to meet our general needs:

Cache

Cache Middleware

If the middleware is enabled, Django willCACHE_MIDDLEWARE_SECONDSThe configured parameters are cached at the whole site level.

Common

General Middleware

This middleware provides us with some convenient functions:

  • prohibitDISALLOWED_USER_AGENTSUser agent access server in
  • Automatically add slash suffix and WWW prefix for URL. If the configuration itemAPPEND_SLASHbyTrue, and the URL to be accessed does not have a slash suffixURLconfIf there is no successful match in, a slash will be added automatically, and then match again. If the match is successful, jump to the corresponding URL.PREPEND_WWWThe functions of are similar.
  • Set for non flow responseContent-LengthHeader information.

As an example, the source code is additionally posted here, located atdjango.middleware.commonIn the module, it is relatively simple and easy to read and understand:

class CommonMiddleware(MiddlewareMixin):
    """
    Doc removed
    """
    response_redirect_class = HttpResponsePermanentRedirect

    def process_request(self, request):
        # Check for denied User-Agents
        if 'HTTP_USER_AGENT' in request.META:
            for user_agent_regex in settings.DISALLOWED_USER_AGENTS:
                if user_agent_regex.search(request.META['HTTP_USER_AGENT']):
                    raise PermissionDenied('Forbidden user agent')

        # Check for a redirect based on settings.PREPEND_WWW
        host = request.get_host()
        must_prepend = settings.PREPEND_WWW and host and not host.startswith('www.')
        redirect_url = ('%s://www.%s' % (request.scheme, host)) if must_prepend else ''

        # Check if a slash should be appended
        if self.should_redirect_with_slash(request):
            path = self.get_full_path_with_slash(request)
        else:
            path = request.get_full_path()

        # Return a redirect if necessary
        if redirect_url or path != request.get_full_path():
            redirect_url += path
            return self.response_redirect_class(redirect_url)

    def should_redirect_with_slash(self, request):

        if settings.APPEND_SLASH and not request.path_info.endswith('/'):
            urlconf = getattr(request, 'urlconf', None)
            return (
                not is_valid_path(request.path_info, urlconf) and
                is_valid_path('%s/' % request.path_info, urlconf)
            )
        return False

    def get_full_path_with_slash(self, request):

        new_path = request.get_full_path(force_append_slash=True)
        if settings.DEBUG and request.method in ('POST', 'PUT', 'PATCH'):
            raise RuntimeError(
                "You called this URL via %(method)s, but the URL doesn't end "
                "in a slash and you have APPEND_SLASH set. Django can't "
                "redirect to the slash URL while maintaining %(method)s data. "
                "Change your form to point to %(url)s (note the trailing "
                "slash), or set APPEND_SLASH=False in your Django settings." % {
                    'method': request.method,
                    'url': request.get_host() + new_path,
                }
            )
        return new_path

    def process_response(self, request, response):
        # If the given URL is "Not Found", then check if we should redirect to
        # a path with a slash appended.
        if response.status_code == 404:
            if self.should_redirect_with_slash(request):
                return self.response_redirect_class(self.get_full_path_with_slash(request))

        if settings.USE_ETAGS and self.needs_etag(response):
            warnings.warn(
                "The USE_ETAGS setting is deprecated in favor of "
                "ConditionalGetMiddleware which sets the ETag regardless of "
                "the setting. CommonMiddleware won't do ETag processing in "
                "Django 2.1.",
                RemovedInDjango21Warning
            )
            if not response.has_header('ETag'):
                set_response_etag(response)

            if response.has_header('ETag'):
                return get_conditional_response(
                    request,
                    etag=response['ETag'],
                    response=response,
                )
        # Add the Content-Length header to non-streaming responses if not
        # already set.
        if not response.streaming and not response.has_header('Content-Length'):
            response['Content-Length'] = str(len(response.content))

        return response

    def needs_etag(self, response):
        """Return True if an ETag header should be added to response."""
        cache_control_headers = cc_delim_re.split(response.get('Cache-Control', ''))
        return all(header.lower() != 'no-store' for header in cache_control_headers)

GZip

Content compression Middleware

It is used to reduce response volume, reduce bandwidth pressure and improve transmission speed.

The middleware must be in front of all other middleware that need to read and write response body content.

The response content will not be compressed if one of the following conditions exists:

  • Content less than 200 bytes
  • Already set upContent-EncodingHead attribute
  • RequestedAccept-EncodingThe header property does not contain gzip

have access togzip_page()Decorator, open gzip compression service separately for view.

Conditional GET

Conditional get access middleware is rarely used.

Locale

Localization Middleware

For dealing with internationalization and localization, language translation.

Message

Message middleware

Cookie based or session based message functions are more commonly used.

Security

Security Middlewares

django.middleware.security.SecurityMiddlewareMiddleware provides us with a series of website security functions. It mainly includes the following, which can be opened or closed separately:

  • SECURE_BROWSER_XSS_FILTER
  • SECURE_HSTS_INCLUDE_SUBDOMAINS
  • SECURE_CONTENT_TYPE_NOSNIFF
  • SECURE_HSTS_PRELOAD
  • SECURE_HSTS_SECONDS
  • SECURE_REDIRECT_EXEMPT
  • SECURE_SSL_HOST
  • SECURE_SSL_REDIRECT

Session

Session middleware is very common.

Site

Site framework.

This is a very useful but neglected function.

It allows your Django to have multi site support.

By adding a site attribute, distinguish the corresponding site that the current request requests to visit.

You don’t need more than one IP or domain name, you don’t need to turn on more than one server, you just need one site attribute, and you can handle multi site services.

Authentication

Certification framework

Django is one of the most important middleware, providing user authentication services.

CSRF protection

Middleware providing CSRF defense mechanism
X-Frame-Options

Click hijack defense Middleware

Custom Middleware

Sometimes, in order to implement some specific requirements, we may need to write our own middleware.

In terms of writing methods, it should be noted that there are two writing methods in the current Django version 2.2. One is the example provided on Django’s current official website, and the other is the old version. In essence, the two ways are the same.

Let’s take a look at the traditional and most frequently used technical articles.

Traditional approach

Five hook functions

The traditional way to customize middleware is to write five hook functions:

  • process_request(self,request)
  • process_response(self, request, response)
  • process_view(self, request, view_func, view_args, view_kwargs)
  • process_exception(self, request, exception)
  • process_template_response(self,request,response)

Any one or more of them can be implemented!

Details of Django Middleware

process_request()

Signature:process_request(request)

The main hook!

There is only one parameter, that isrequestRequest content, andrequestIt’s the same. All middleware is the samerequest, will not change. Its return value can beNoneOr it could beHttpResponseObject. returnNoneIn other words, it means that everything is normal. Continue to go through the process and hand it over to the next middleware for processing. returnHttpResponseObject, it will not continue to execute the subsequent middleware, nor execute the view function, but return the response content to the browser.

process_response()

Signature:process_response(request, response)

The main hook!

There are two parameters,requestandresponserequestIs the content of the request,responseIs returned by the view functionHttpResponseObject. The return value of the method must be aHttpResponseObject, cannot beNone

process_response()Method is executed after the view function is executed and in reverse order of the configuration order.

process_view()

Signature:process_view(request, view_func, view_args, view_kwargs)

  • Request: · httprequest · object.
  • view_ Func: the real business logic view function (not the string name of the function).
  • view_ Args: location parameter list
  • view_ Kwargs: keyword parameter dictionary

Remember:process_view()It is executed before Django calls the real business view, and in positive order. Whenprocess_request()After normal execution, you will enterurlconfIn the routing phase, and find the corresponding view. Before executing the view function, execute theprocess_view()Middleware hook.

This method must returnNoneOr aHttpResponseObject. If you returnNoneDjango will continue to process the current request and execute otherprocess_view()Middleware hook, and finally execute the corresponding view. If you return aHttpResponseObject, Django does not call the business view, but executes the response middleware and returns the result.

process_exception()

Signature:process_exception(request, exception)

  • request:HttpRequestobject
  • Exception: the specific exception object raised by the view function

When a view throws an exception during execution, Django will automatically call theprocess_exception()method.process_exception()Or return to oneNone, or return oneHttpResponseObject. If you returnHttpResponseObject, template response and response middleware will be called, otherwise normal exception handling process will be carried out.

Similarly, each middleware is called in reverse orderprocess_exceptionMethod to short circuit the mechanism.

process_template_response()

Signature:process_template_response(request, response)

  • request:HttpRequestobject
  • response : TemplateResponseobject

process_template_response()Method is called after the business view is executed.

Normally, after a view is executed, a template will be rendered and returned to the user as a response. With this hook method, you can reprocess the process of rendering the template and add the business logic you need.

aboutprocess_template_response()Methods are also executed in reverse order.

Hook method execution process

(Note: all the pictures are from the Internet, and are deleted!)
An ideal middleware execution process may onlyprocess_request()andprocess_response()The procedure is as follows:

Details of Django Middleware

Once any middleware returns aHttpResponseObject, enter the response process immediately! It should be noted that the response hook method of the unimplemented middleware will not be executed either. This is a short-circuit or peeling process.

If anyprocess_viewThe intervention of methods will become the following:
Details of Django Middleware

The overall execution process and mechanism are as follows:

Details of Django Middleware

A careful study of the following execution process can deepen your understanding of middleware.

Details of Django Middleware

Example demonstration

After introducing the theory, let’s demonstrate it with practical examples.

It should be noted that the traditional method is called because a parent class that will be discarded in the future is imported here, that is:

from django.utils.deprecation import MiddlewareMixin

“Depreciation” means abandonment, devaluation, depreciation and opposition, that is to say, thisMiddlewareMixinClass should be deleted in the future!

Let’s take a lookMiddlewareMixinSource code:

class MiddlewareMixin:
    def __init__(self, get_response=None):
        self.get_response = get_response
        super().__init__()

    def __call__(self, request):
        response = None
        if hasattr(self, 'process_request'):
            response = self.process_request(request)
        if not response:
            response = self.get_response(request)
        if hasattr(self, 'process_response'):
            response = self.process_response(request, response)
        return response

This class does not define five hook methods, but defines__call__Method, byhasattrReflection of searchprocess_requestIf the hook function exists, execute it. Its essence is the same as the example provided by Django official website later, that is, the new writing method!

Now, suppose we have an app calledmidware, create amiddleware.pyModule, write the following code:

from django.utils.deprecation import MiddlewareMixin


class Md1(MiddlewareMixin):

    def process_request(self,request):
        Print ("MD1 process request")

    def process_response(self,request,response):
        Print ("MD1 returns response")
        return response

    def process_view(self, request, view_func, view_args, view_kwargs):

        Print ("MD1 before executing% s view"% view_ func.__ name__ )

    def process_exception(self,request,exception):
        Print ("MD1 handle view exception...")



class Md2(MiddlewareMixin):

    def process_request(self,request):
        Print ("MD2 process request")

    def process_response(self,request,response):
        Print ("MD2 returns response")
        return response

    def process_view(self, request, view_func, view_args, view_kwargs):

        Print ("MD2 before executing% s view"% view_ func.__ name__ )

    def process_exception(self,request,exception):
        Print ("MD2 handle view exception...")

Then, we cansetting.pyThese two customized middleware are configured in:

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
    'midware.middleware.Md1',
    'midware.middleware.Md2',
]

staymidware/views.pyCreate a simple view in:

from django.shortcuts import render, HttpResponse


def mid_test(request):
    Print ('execute view mid_ test')
    # raise
    return HttpResponse('200,ok')

Of whichraiseCan be used to testprocess_exception()Hook.

Write aurlconfFor testing views, such as:

from midware import views as mid_views

urlpatterns = [
    path('midtest/', mid_views.mid_test),
]

Restart server, access...../midtest/, you can see the following information in the console:

MD1 processing request
MD2 processing request
MD1 is executing mid_ Before test view
MD2 is executing mid_ Before test view
Executive view mid_ test
MD2 return response
MD1 return response

Django official method

In Django’s official documentation (currently 2.2), we can see a completely different way of writing.

It’s not written in this wayprocess_request()andprocess_response()Methods are written to integrate them directly.

This way is recommended by the government!

Middleware is essentially a callable object (function, method, class), which accepts a request(request), and returns a response(response)OrNone, just like a view. Its initialization parameter is a parameter namedget_responseThe callable object of.

Middleware can be written as the following functions (the following syntax, essentially a python decorator, is not recommended):

def simple_middleware(get_response):
    #Configuration and initialization

    def middleware(request):

        #Write the specific business view here and the code to be executed before the subsequent middleware is called

        response = get_response(request)

        #Write the code to be executed after the view call here

        return response

    return middleware

Or write a class (true. Recommended form). The instance of this class is callable, as follows:

class SimpleMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
         #Configuration and initialization

    def __call__(self, request):

        #Write the code to be executed before the view and subsequent middleware are called here
        #This is actually the old process_ Code for the request () method

        response = self.get_response(request)

        #Write the code to be executed after the view call here
        #This is actually the old process_ Code for the response () method

        return response

(is it feeling and frontMiddlewareMixinClass is very similar?)

Provided by Djangoget_responseThe method may be an actual view (if the current middleware is the last one listed), or the next Middleware in the list. We don’t need to know or care about what it is, it just represents the next step.

Two considerations:

  • Django only usesget_responseParameter initialization middleware, so it cannot be__init__()Add additional parameters.
  • And every request__call__()When the web server is started,__init__()Called only once.

Example demonstration

We just need to put theMd1andMd2Two middleware classes can be modified to the following code:

class Md1:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):

        Print ("MD1 process request")

        response = self.get_response(request)

        Print ("MD1 returns response")

        return response

    def process_view(self, request, view_func, view_args, view_kwargs):

        Print ("MD1 before executing% s view"% view_ func.__ name__ )

    def process_exception(self,request,exception):
        Print ("MD1 handle view exception...")


class Md2:
    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        Print ("MD2 process request")

        response = self.get_response(request)

        Print ("MD2 returns response")

        return response

    def process_view(self, request, view_func, view_args, view_kwargs):
        Print ("MD2"% view before executing% s view_ func.__ name__ )

    def process_exception(self, request, exception):
        Print ("MD2 handle view exception...")

As you can see, we no longer need inheritanceMiddlewareMixinClass.

The actual execution results are the same.

Application example 1: IP interception

If we want to restrict some IP access to the server, we cansettings.pyAdd aBLACKLIST(all uppercase) list, where the restricted IP address is written.

Then, we can write the following middleware:

from django.http import HttpResponseForbidden
from django.conf import settings

class BlackListMiddleware():

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):

        if request.META['REMOTE_ADDR'] in getattr(settings, "BLACKLIST", []):
            Return httpresponseforbidden ('< H1 > the IP address is restricted! </h1>')

        response = self.get_response(request)

        return response

The specific middleware registration, view and URL will not be described in detail.

Application example 2: debug page

After the website goes online, we willDEBUGChange toFalse, which is safer. But the server5xxWhen a series of errors occur, the administrator cannot see the details of the errors, which makes debugging inconvenient. Is there any way to solve this problem more conveniently?

  • Normal visitors see 500 error pages
  • The administrator sees the error details debug page

You can do it with middleware! The code is as follows:

import sys
from django.views.debug import technical_500_response
from django.conf import settings


class DebugMiddleware():

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):

        response = self.get_response(request)

        return response

    def process_exception(self, request, exception):
        #If it is an administrator, a special response object is returned, that is, the debug page
        #If it is a normal user, it will return to none and give it to the default process
        if request.user.is_superuser or request.META.get('REMOTE_ADDR') in settings.ADMIN_IP:
            return technical_500_response(request, *sys.exc_info())

ThroughifJudge whether the current login user is super administrator or whether the IP address of the current user is in the administrator IP address list. One of the two is to judge that the current user has permission to view the debug page.

Next register the middleware and add a line to the test viewraise。 Revise againsettings.py, willDebugSet toFalse, provideALLOWED_HOSTS = ["*"], settings such asADMIN_IP = ['192.168.0.100'], and then start the server0.0.0.0:8000Test the middleware from different LAN IP.

Normally, the administrator should see a debug page similar to the following:

RuntimeError at /midtest/
No active exception to reraise
Request Method: GET
Request URL:    http://192.168.0.100:8000/midtest/
Django Version: 2.0.7
Exception Type: RuntimeError
Exception Value:    
No active exception to reraise
.....

Ordinary users can only see:

A server error occurred.  Please contact the administrator.