Django 1.8 official document translation: 13-3 log

Time:2021-6-13

journal

Log quick start

Django uses Python built-inloggingThe module prints the log. The use of this module is discussed in detail in Python’s own documentation. If you’ve never used Python’s logging framework (or even used it), see the quick introduction below.

Composition of logging

Python’s logging configuration consists of four parts

Loggers

The logger is the entry of the log system. Each logger is a named container to which messages to be processed can be written.

Every logger has one_ Log level. The log level indicates the severity of the message that the logger will process. Python defines the following logging levels:

  • DEBUG: underlying system information for debugging purposes

  • INFO: general system information

  • WARNING: indicates a minor problem.

  • ERROR: indicates a major problem.

  • CRITICAL: indicates a fatal problem.

Every message written to the logger is a_ Logging. Each log record also has a_ Log level, It represents the severity of the corresponding message. Each log record can also contain useful meta information describing the event being printed. These meta information can contain many details, such as backtracking stack or error code.

When a message is given to the logger, the log level of the message is compared with the log level of the logger. If the log level of a message is greater than or equal to the log level of the logger, the message will continue processing. If less than, the message is ignored.

Once the logger decides that a message needs to be processed, it passes it to a_ Handler_。

Handlers

The handler decides how to handle each message in the logger. It represents a specific logging behavior, such as writing messages to the screen, to a file, or to a network socket.

Like the logger, the handler has a logging level. If the log level of a message is less than that of the handler, the handler ignores the message.

A logger can have multiple handlers, and each handler can have a different log level. In this way, different forms of processing can be provided according to the importance of the message. For example, you can use a handler toERRORandCRITICALMessages are sent to a page service, and all messages (includingERRORandCRITICALMessages) are recorded in a file for later analysis.

Filters

The filter is used for additional control over the logging passed from the logger to the handler.

By default, any message that meets the log level will be processed. By installing a filter, you can add additional conditions to log processing. For example, you can install a filter that only processes data from a specific sourceERRORNews.

Filters can also be used to modify the priority of the log records to be processed. For example, if the logging meets certain conditions, you can write a filter to remove the logging from theERRORDown toWARNING

Filters can be installed on the logger or handler; Multiple filters can be connected in series to achieve multi-layer filter behavior.

Formatters

Finally, the logging needs to be converted to text. Formatter represents the format of the text. Fomatter is usually composed of_ Logging properties_Python format string composition; You can also write a custom fomatter to implement your own format.

Using logging

After you configure logger, handler, filter, and formatter, you need to put logging calls into your code. Using the logging framework is very simple. Here’s an example:

# import the logging library
import logging

# Get an instance of a logger
logger = logging.getLogger(__name__)

def my_view(request, arg1, arg):
    ...
    if bad_mojo:
        # Log an error message
        logger.error('Something went wrong!')

this is it! Every timebad_mojoCondition, an error log record will be written.

Name logger

logging.getLogger()Call to get (if necessary, create) an instance of logger. The logger instance is identified by name. Logger uses a name to identify its configuration.

The name of logger is usually used__name__, which is the name of the python module that contains the logger. This allows you to log calls based on the module filter and handle. If you want to organize log messages in other ways, you can provide a point separated name to identify your logger

# Get an instance of a specific named logger
logger = logging.getLogger('project.interesting.stuff')

Point separated logger names define a hierarchy.project.interestingLogger is considered to beproject.interesting.stuffThe upper level of the logger;projectLogger isproject.interestingOne level above the logger.

Why is hierarchy so important? Because you can set the logger_ Communication_ Their logging calls go up one level. In this way, you can define a series of handlers on the root logger and capture all logging calls in the child logger. stayprojectThe handler defined in theproject.interestingandproject.interesting.stuffAll log messages on the logger.

This propagation behavior can be controlled based on each logger. If you don’t want a logger to propagate messages to its upper level, you can turn off this behavior.

Logging call

The logger instance provides an entry method for each default log level

  • logger.debug()

  • logger.info()

  • logger.warning()

  • logger.error()

  • logger.critical()

There are two other calls:

  • logger.log(): manually specify the log level when printing messages.

  • logger.exception(): create aERRORLevel log message, which encapsulates the frame of the current exception stack.

Configure logging

Of course, just putting the logging call into your code is not enough. You also need to configure logger, handler, filter and formatter to make sure that the output of the log is meaningful.

Python’s logging library provides several techniques for configuring logging, from program interfaces to configuration files. By default, Django usesDictconfig format

To configure logging, you need to useLOGGINGTo define the logging settings in dictionary form. These settings describe the logger, handler, filter and formatter of your logging settings, as well as their log level and other properties.

By default,LOGGINGSettings and_ Default logging configuration of Django_Merge.

IfLOGGINGIndisable_existing_loggersKey isTrue(default), then all loggers in the default configuration will be disabled. The disable and delete of logger are different; The logger still exists, but any information passed to it will be discarded silently, and it will not be propagated to the higher level logger. So, you should use it very carefully'disable_existing_loggers': True; It may not be what you want. You can set it updisable_existing_loggersbyFalseAnd redefine some or all of the default loggers; Or you can set itLOGGING_CONFIGbyNone, and_ Handle logging configuration by yourself_

The configuration of logging belongs to Djangosetup()Part of a function. So you can be sure that logger is always available in your project code.

Examples

Dictconfig formatThe complete document of is the best information source of logging dictionary configuration. But to give you a taste, here are a few examples.

First, here’s a simple configuration that will come from_django.request_All log requests of logger are written to a local file:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'file': {
            'level': 'DEBUG',
            'class': 'logging.FileHandler',
            'filename': '/path/to/django/debug.log',
        },
    },
    'loggers': {
        'django.request': {
            'handlers': ['file'],
            'level': 'DEBUG',
            'propagate': True,
        },
    },
}

If you use this example, make sure you modify it'filename'The path is a location that the user running the Django application has permission to write to.

Second, the following example shows how to make the logging system print Django’s logs to the console.django.requestanddjango.securityThe log will not be propagated to the next level. It can be useful during local development.

By default, this configuration will onlyINFOAnd higher level logs are sent to the console. There is not much such log information in Django. You can set environment variablesDJANGO_LOG_LEVEL=DEBUGTake a look at Django’s debug log, which contains all the database queries, so it’s very detailed.

import os

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'handlers': {
        'console': {
            'class': 'logging.StreamHandler',
        },
    },
    'loggers': {
        'django': {
            'handlers': ['console'],
            'level': os.getenv('DJANGO_LOG_LEVEL', 'INFO'),
        },
    },
}

Finally, here’s a fairly complex logging setting:

LOGGING = {
    'version': 1,
    'disable_existing_loggers': False,
    'formatters': {
        'verbose': {
            'format': '%(levelname)s %(asctime)s %(module)s %(process)d %(thread)d %(message)s'
        },
        'simple': {
            'format': '%(levelname)s %(message)s'
        },
    },
    'filters': {
        'special': {
            '()': 'project.logging.SpecialFilter',
            'foo': 'bar',
        }
    },
    'handlers': {
        'null': {
            'level': 'DEBUG',
            'class': 'logging.NullHandler',
        },
        'console': {
            'level': 'DEBUG',
            'class': 'logging.StreamHandler',
            'formatter': 'simple'
        },
        'mail_admins': {
            'level': 'ERROR',
            'class': 'django.utils.log.AdminEmailHandler',
            'filters': ['special']
        }
    },
    'loggers': {
        'django': {
            'handlers': ['null'],
            'propagate': True,
            'level': 'INFO',
        },
        'django.request': {
            'handlers': ['mail_admins'],
            'level': 'ERROR',
            'propagate': False,
        },
        'myproject.custom': {
            'handlers': ['console', 'mail_admins'],
            'level': 'INFO',
            'filters': ['special']
        }
    }
}

This logging configuration does the following:

  • Parse the configuration in the format of ‘dictconfig version 1’. So far, this is the only version of dictconfig format.

  • Define two formatters:

    • simple, which only outputs the level of the log (for example,DEBUG)And log messages.

      `The format 'string is a normal Python formatted string that describes the details of each line of log. The full details of the output can be found in the [formatter document]( https://docs.python.org/library/logging.html#formatter -Objects).
    • verboseIt outputs log level, log message, time, process, thread and the module that generates log message.

  • Define filter——project.logging.SpecialFilter, and use aliasesspecial. If additional parameters are required in the construction of the filter, they can be provided with additional keys in the configuration field of the filter. In this case, in the instantiationSpecialFilterWhen,fooThe value of the parameter is usedbar

  • Three handlers are defined

    • null, a nullhandler, which passesDEBUG(and higher) message to/dev/null

    • console, a streamhandler that will printDEBUG(and more advanced) messages to stderr. This handler usessimpleOutput format.

    • mail_admins, an adminemail handler, which will be sent by mailERROR(and more advanced) messages to the site administrator. This handler usesspecial filter。

  • Configure three loggers:

    • djangoIt passes all the informationINFOAnd more advanced information for younull handler。

    • django.requestIt passes all the informationERRORMessage tomail_adminshandler。 In addition, mark the loggernoSpread the news up. This means writedjango.requestLog information for will not be useddjangoLogger processing.

    • myproject.customIt passes all the informationINFOAnd more advanced messages and throughspecialFilter message to two handlers——consoleandmail_admins. It means allINFO(and more advanced) messages will be printed to the console;ERRORandCRITICALMessages are also sent by email.

Custom logging configuration

If you don’t want to configure the logger using Python’s dictconfig format, you can specify your own configuration mode.

LOGGING_CONFIGSetting defines a callable object that is used to configure Django’s logger. By default, it points to Python’slogging.config.dictConfig()Function. However, if you want to use a different configuration procedure, you can use other callable objects that take only one parameter. When configuring logging, theLOGGINGAs the value of the parameter.

Disable logging configuration

If you don’t want to configure logging at all (or you want to configure logging manually in your own way), you can set itLOGGING_CONFIGbyNone. This will be disabled_ Django default logging_The configuration process of. The following example disable the logging configuration of Django, and then configure logging manually:

settings.py

LOGGING_CONFIG = None

import logging.config
logging.config.dictConfig(...)

set upLOGGING_CONFIGbyNoneIt only means to disable the automatic configuration process, not logging itself. If you disable the configuration process, Django still performs the logging call, except that it calls the logging behavior defined by default.

Django’s logging extensions

Django provides many tools to handle the unique logging requirements in a web server environment.

Loggers

Django provides several built-in loggers.

django

djangoIt’s a logger that captures all the information. Messages are not submitted directly to the logger.

django.request

Records messages related to processing requests. 5xx response asERRORNews; 4xx response asWARNINGNews.

This logger’s message has the following additional context:

  • status_code: HTTP response code of the request.

  • request: request object to generate log information.

django.db.backends

Messages related to the code that interacts with the database. For example, an HTTP request to execute an application level SQL statement willDEBUGLevel is logged to the logger.

This logger’s message has the following additional context:

  • duration: time spent executing SQL statements.

  • sql: the SQL statement to execute.

  • params: parameter used in SQL call.

Due to performance reasons, SQL logs are only available inset upThen turn it on. Debug set toTrueRegardless of the log level or the processor installed.

The log here does not contain frame level initialization (for example,SET TIMEZONE)And transaction management queries (for example,BEGINCOMMITandROLLBACK)。 If you want to see all the database queries, you can open the query log in the database.

django.security.*

Security logger will receive any message that appearsSuspiciousOperationI’m sorry to hear from you. Every subtype of suspicious operation has a child logger. The level of the log depends on the location of the exception handling. Most of the time it’s a warning log, and ifSuspiciousOperationWhen it arrives at the WSGI handler, it is recorded as an error. For example, if the request contains httpHostHead andALLOWED_HOSTSIf not, Django will return a 400 response and record an error message todjango.security.DisallowedHost logger。

By default, only thedjango.securityAll other child loggers will be propagated to the upper level logger.django.securityConfiguration and implementation of loggerdjango.requestAny error message will be sent to the site administrator by email. becauseSuspiciousOperationThe request that caused the 400 response will not be sent in thedjango.requestLog in the logger, but only in thedjango.securityLog in the logger.

To silently discard a certain type of suspicious operation, you can override its logger as follows:

'loggers': {
    'django.security.DisallowedHost': {
        'handlers': ['null'],
        'propagate': False,
    },
},

django.db.backends.schema

New in Django 1.7\.

When_ Migration framework_These SQL queries are recorded when the SQL queries executed will change the schema of the database. Note that it doesn’t recordRunPythonQuery executed.

Handlers

Django provides another Handler Based on the handler provided by Python logging module.

_class _AdminEmailHandler(_include_html=False_, _email_backend=None_)[source]

The handler sends each log information it receives to the site administrator by email.

If the log record containsrequestProperty, the full details of the request will be included in the message.

If the log record contains stack backtracking information, the stack backtracking will also be included in the message.

AdminEmailHandlerOfinclude_htmlParameter is used to control whether the message contains HTML attachments, which containDEBUGbyTrueThe full page of the. To set this value in the configuration, you can include it in thedjango.utils.log.AdminEmailHandlerIn the definition of handler, as follows:

'handlers': {
    'mail_admins': {
        'level': 'ERROR',
        'class': 'django.utils.log.AdminEmailHandler',
        'include_html': True,
    }
},

Note that the HTML in the email contains the complete backtracking stack, including the names and values of local variables at each level of the stack, as well as your Django settings. This information can be very sensitive and you may not want to send it by email. At this point, you can consider using a similar methodSentryThis kind of thing, the complete information and security information of the backtracking stack_ No_ Send by email. You can also explicitly filter out specific sensitive information from error reports – see_ Filtering error reports_

By settingAdminEmailHandlerOfemail_backendParameter, which can override the_email backend_Like this:

'handlers': {
    'mail_admins': {
        'level': 'ERROR',
        'class': 'django.utils.log.AdminEmailHandler',
        'email_backend': 'django.core.mail.backends.filebased.EmailBackend',
    }
},

By default, theEMAIL_BACKENDThe message backend specified in.

send_mail(_subject_, _message_, _args_, _*kwargs_)[source]

New in Django 1.8\.

Send email to administrator users. If you want to customize its behavior, you can subclass itAdminEmailHandlerClass and override this method.

Filters

Django provides two filters in addition to the filters provided by the python logging module.

_class _CallbackFilter(_callback_)[source]

The filter takes a callback function (which takes a single parameter, that is, something to record) and calls it for each record passed to the filter. If the callback function returns false, the record will not be processed.

For example, to filter from admin mailUnreadablePostError(only generated when the user cancels uploading), you can create a filter function:

from django.http import UnreadablePostError

def skip_unreadable_post(record):
    if record.exc_info:
        exc_type, exc_value = record.exc_info[:2]
        if isinstance(exc_value, UnreadablePostError):
            return False
    return True

Then add it to the configuration of the logger:

'filters': {
    'skip_unreadable_posts': {
        '()': 'django.utils.log.CallbackFilter',
        'callback': skip_unreadable_post,
    }
},
'handlers': {
    'mail_admins': {
        'level': 'ERROR',
        'filters': ['skip_unreadable_posts'],
        'class': 'django.utils.log.AdminEmailHandler'
    }
},

_class _RequireDebugFalse[source]

This filter only passes records after it is set. Debug is false.

This filter follows theLOGGINGDefault configuration to ensureAdminEmailHandlerOnly inDEBUGbyFalseSend the wrong email when you send it.

'filters': {
    'require_debug_false': {
        '()': 'django.utils.log.RequireDebugFalse',
    }
},
'handlers': {
    'mail_admins': {
        'level': 'ERROR',
        'filters': ['require_debug_false'],
        'class': 'django.utils.log.AdminEmailHandler'
    }
},

_class _RequireDebugTrue[source]

This filter is similar toRequireDebugFalse, except that the record is only inDEBUGbyTrueThis is the case when the message is delivered.

Django’s default logging configuration

By default, the logging configuration of Django is as follows:

WhenDEBUGbyTrueTime:

  • djangoThe global logger of sends a level equal to or advanced to the consoleINFOAll of the news. Django does not make any log calls at this time (all in theDEBUGLevel log, or bydjango.requestanddjango.securityLog processed).

  • py.warningsLogger, which handles thewarnings.warn()The message will be sent to the console.

WhenDEBUGbyFalseTime:

  • django.requestanddjango.securityLoggers toAdminEmailHandlerSend withERRORorCRITICALLevel message. These loggers ignore any level equal to or less thanWARNINGThe logged logs are not passed to other loggers (they are not passed todjangoGlobal logger, even ifDEBUGbyTrue)。

See also_ Configuration log_To learn how to supplement or replace the default log configuration.

translator:Django document collaborative translation teamOriginal text:Logging

This paper is based onCC BY-NC-SA 3.0Please keep the author’s signature and the source of the article.

Django document collaborative translation teamWe are short of staff. Interested friends can join us. It is totally public welfare. Communication group: 467338606.