A Logging System for Python

A Logging System for Python

A Logging System for Python	Home Download Copyright & License Recent Changes API Documentation
"Oh, I'm a lumberjack and I'm okay..." (Monty Python, The Lumberjack Song)

Table of Contents

Abstract

Motivation

Influences

A Simple Example

Control Flow

Levels

Loggers

Handlers

Formatters

Filters

Configuration

The GUI Configurator

Case Scenarios

Thread Safety

On-The-Fly Reconfiguration

Module-Level Convenience Functions

Performance

Implementation Status

Acknowledgements

Still To Do

Download and Installation

Change History

Copyright and License

Abstract

There was a need for a standard logging system in Python, as comprehensively documented
in PEP 282 and enthusiastically
endorsed by the BDFL in the Parade
of the PEPs. By a happy coincidence, the package described here was already in
development and fairly close in intent and design to the description in the aforementioned
PEP, borrowing as it did heavily from JSR-47 (now JDK 1.4's java.util.logging package) and
log4j. This page describes it
in more detail. As I have tweaked the package to meet comments on PEP 282, I have structured
this page in the same way as the original PEP. This package is now part of Python 2.3, but if
you have an earlier version of Python, you can download the package from here and use it
with Python versions between 1.5.2 and 2.2.x.

Motivation

The Python community has been incredibly helpful to me, a relative newcomer to the
language. Python and its community has certainly saved me much time and effort, and it
seems appropriate to give something back to the community by offering up this package for
people to try. Any feedback will be
gratefully accepted.

Influences

This package owes its greatest debt to Apache log4j. Due notice was also taken
of log4j's comprehensive critique (no longer online) of JSR47. This package bears a close
resemblance to log4j, but is not a close translation. I have attempted to be more minimalist
(and hopefully more Pythonic) in my approach. You be the judge!

A Simple Example

Using the package doesn't get much simpler. It is packaged as a standard Python package
called (unsurprisingly) logging. You just need to import logging
and you're ready to go. Minimal example:

# --- app.py --------------------------------------------------------------------
import logging

logging.warn("Hello")
logging.error("Still here...")
logging.warn("Goodbye")

When you run app.py, the results are:

WARNING:root:Hello
ERROR:root:Still here...
WARNING:root:Goodbye

Don't worry about the format of the output - it's all configurable. Here's a slightly
more involved example; if you've just looked at PEP 282 you will probably get a feeling of
dej� vu. (This is intentional.)

# --- mymodule.py --------------------------------------------------------------------
import logging
log = logging.getLogger("MyModule")

def doIt():
    log.debug("doin' stuff")
    #do stuff...but suppose an error occurs?
    raise TypeError, "bogus type error for testing"

# --- myapp.py -----------------------------------------------------------------------
import logging, mymodule

logging.basicConfig()

log = logging.getLogger("MyApp")
log.setLevel(logging.DEBUG) #set verbosity to show all messages of severity >= DEBUG
log.info("Starting my app")
try:
    mymodule.doIt()
except Exception, e:
    log.exception("There was a problem.")
log.info("Ending my app")

When you run myapp.py, the results are:

INFO:MyApp:Starting my app
ERROR:MyApp:There was a problem.
Traceback (most recent call last):
  File "myapp.py", line 9, in ?
    mymodule.doIt()
  File "mymodule.py", line 7, in doIt
    raise TypeError, "Bogus type error for testing"
TypeError: Bogus type error for testing
INFO:MyApp:Ending my app

But don't worry - the above output is not hardcoded into the package. It's just an
example of what you can do with very little work. As you can see, exceptions are handled
as one would expect.

Control Flow

The package pretty much matches the PEP regarding control flow. The user of the package
makes logging calls on instances of Logger, which are organized into a
hierarchy based on a "dotted name" namespace. This hierarchy is embodied in an
encapsulated singleton Manager instance (which can be ignored by users of the
package, for most purposes). Based on the type of logging call and the logging
configuration (see below), the call may be passed through a set of Filter
instances to decide whether it should be dropped. If not, then the logger consults a set
of Handler instances which are associated with it, and asks each handler
instance to "handle" the logging event. By default, the system moves up the
namespace hierarchy and invokes handlers on all loggers at or above the level of the
logger on which the logging call was made. (You can override this by setting a logger's
"propagate" attribute to 0 - no traversal up the hierarchy is made from such a
logger. But I'm getting ahead of myself...)

Handlers are passed LogRecord instances which (should) contain all the
information we're interested in logging. Handlers, too, can invoke filters to determine
whether a record should be dropped. If not, then the handler takes a handler-specific
action to actually log the record to a file, the console or whatever.

Levels

The following levels are implemented by default:

DEBUG
INFO
WARNING
ERROR
CRITICAL

The CRITICAL level replaces the earlier FATAL level. You can
use either (for now), but CRITICAL is preferred since FATAL
implies that the application is about to terminate. This is not true for many systems
which use logging - for example, a Web server application which encounters a CRITICAL
condition (e.g. running out of resources) will still try to keep going as best it can.

FATAL (and the corresponding fatal() methods) may be removed
in future versions of the package. Currently, CRITICAL is synonymous with FATAL
and critical() methods are synonymous with fatal().

Exceptions logged via exception() use the ERROR level for
logging. If it is desired to log exception information with arbitrary logging levels, this
can be done by passing a keyword argument exc_info with a true value to the
logging methods (see the API documentation for more details).

The levels are not deeply hardcoded into the package - the number of levels, their
numeric values and their textual representation are all configurable. The above levels
represent the experience of the log4j community and so are provided as the default levels
for users who do not have very specific requirements in this area.

The example script log_test4.py shows the use of bespoke logging levels
(as well as filtering by level at logger and handler, as well as use of filter classes).