SummarisingLogger
Note
Throughout the examples below, mail messages sent using
smtplib
are printed to the screen so we can see what’s going on:
>>> import smtplib
>>> server = smtplib.SMTP('localhost')
>>> server.sendmail('from@example.com', ['to@example.com'], 'The message')
sending to ['to@example.com'] from 'from@example.com' using ('localhost', 25)
The message
SummarisingLogger
is a handler for the python logging
framework that accumulates log entries and sends a single email
containing all the log entries using an SMTP server when its
close()
method is called.
This close()
method is, by default,
registered as an atexit
function so that the summary mail will
get sent regardless of whether an explicit call is made to the
SummarisingLogger.close()
method.
SummarisingLogger
handlers can be very useful for batch
processes that are frequently run and where people would like an email
summary of how the batch run went.
They are configured as any other logging
handler would be, full
details of which can be found in the Python core
documentation. For the examples below, we’ll stick to manually
configuring the logging elements.
A SummarisingLogger
is instantiated as follows:
>>> import logging
>>> from mailinglogger import SummarisingLogger
>>> handler = SummarisingLogger('from@example.com',('to@example.com',))
It can then be added as a handler for any logger as follows:
>>> import logging
>>> logger = logging.getLogger()
>>> logger.addHandler(handler)
However, when we log a message, nothing appears to happen:
>>> logging.debug('some debugging')
>>> logging.info('some information')
>>> logging.warning('a warning')
>>> logging.error('my message')
This is because the messages have been recorded and will be sent as a summary when the logging framework is shut down or, by default, when the script that calls the logging function exits.
If we manually close our log handler, we can see the mail gets sent:
>>> handler.close()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
a warning
my message
The logging on script exit is done using python’s atexit
module. Here’s the handler registered above:
>>> print(atexit_handlers)
[<bound method SummarisingLogger.close of <...>>]
Now, to continue with the examples, just like any other handler, we can also set the logging level, which will filter out messages logged below the level set:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',))
>>> logger.addHandler(handler)
>>> handler.setLevel(logging.CRITICAL)
>>> logging.error('an error')
>>> handler.setLevel(logging.WARNING)
>>> logging.warning('a warning')
>>> handler.close()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (WARNING)
To: to@example.com
X-Log-Level: WARNING
X-Mailer: MailingLogger...
a warning
As with MailingLogger
, you can see from the above examples
that SummarisingLogger
sends mail messages that are correctly
formatted, including Date
and Message-ID
headers.
You will also notice that an X-Mailer
header has been added
specifying that mailinglogger
is the sender of the mail.
An X-Log-Level
header has also been added indicating the highest
level message that has been handled by the SummarisingLogger
.
These headers can be useful for filtering mail
sent by MailingLogger
. If you wish to filter mail by
environment or other configuration data, the support for adding
extra headers may be useful.
Avoiding the atexit
handler
In the event you wish to manually call the
close()
method of the handler or use the
logging framework’s shutdown()
functionality rather
than registering an atexit
function, you can create a
SummarisingLogger
and specify that no atexit
function
should be registered:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... atexit=False)
>>> logger.addHandler(handler)
Now, we can see that no atexit
function has been registered:
>>> print(atexit_handlers)
[]
With this configuration, if an entry is logged, the logging framework must be manually shut down for the mail to be sent:
>>> logging.error('my message')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
my message
Because the users of SummarisingLogger
may not have control
over when or how often the logging handlers they configure are closed,
a SummarisingLogger
will not raise exceptions and will not
send duplicate emails if closed more than once:
>>> handler.close()
Likewise, messages logged to the handler after it has been closed will not result in errors but will also not result in emails being sent:
>>> logging.error('my message')
Controlling the subject line
The subject for the summary mail sent is controlled by the subject
parameter to the SummarisingLogger
parameter.
This can be set to a fixed value:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... subject='My Logging Summary')
>>> logger.addHandler(handler)
>>> logging.error('a message')
>>> handler.close()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: My Logging Summary
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
a message
It can also be set using any of the substitution variables described in the SubjectFormatter documentation, for example:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... subject='[%(hostname)s] %(levelname)s - %(line)s')
>>> logger.setLevel(logging.INFO)
>>> logger.addHandler(handler)
>>> logging.info('a message')
>>> logging.error('an error')
>>> handler.close()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: [host.example.com] ERROR - a message
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
a message
an error
You’ll notice that the %(line)
substitution inserts the first line
of the whole summary mail when used with a SummarisingLogger
.
You’ll also notice that the %(levelname)
substitution inserts the
name of the highest level logged while the SummarisingLogger
was active.
If no messages have been handled by the logger, then %(levelname)s
will be the string NOTSET
:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... subject='[%(levelname)s] summary')
>>> logger.addHandler(handler)
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: [NOTSET] summary
To: to@example.com
X-Log-Level: NOTSET
X-Mailer: MailingLogger...
Formatting messages in the body of the summary email
You may also be wondering how you control the formatting of the
messages included in the summary email. This is done using the
standard setFormatter()
method of python log
handlers.
Here’s an example:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',))
>>> handler.setFormatter(logging.Formatter('%(asctime)s [%(levelname)s] %(message)s'))
>>> logger.addHandler(handler)
To show things working, some entries need to be logged. Here’s one at
2007-01-01 10:00:00
:
>>> logging.warning('something happened')
Here’s another at 2007-01-01 12:34:56
:
>>> try:
... raise RuntimeError('badness')
... except:
... logging.error('bad things happened',exc_info=True)
The following shows the mail that would be sent:
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
2007-01-01 10:00:00,000 [WARNING] something happened
2007-01-01 12:34:56,000 [ERROR] bad things happened
Traceback (most recent call last):
...
RuntimeError: badness
Recording and sending at different levels
In some circumstances, you may want to send a summary email when a
certain log level is reached but, when the summary is sent, you want
the summary to include logging at a lower level. To do this, you would
pass a send_level
to the SummarisingLogger
constructor:
>>> handler = SummarisingLogger('from@example.com', ('to@example.com',),
... send_level=logging.ERROR)
>>> handler.setFormatter(logging.Formatter('%(asctime)s [%(levelname)s] %(message)s'))
>>> logger.addHandler(handler)
>>> logging.info('An info message')
>>> logging.error('Something bad happened')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: Mon, 01 Jan 2007 12:34:56 -0000
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
2007-01-01 12:34:56,000 [INFO] An info message
2007-01-01 12:34:56,000 [ERROR] Something bad happened
Limiting the size of emails
All good MTAs will limit the size of message that they will accept. If your script logs an unexpectedly large number of messages, such as when something goes catastrophically wrong, this may result in no email notification being sent.
More commonly, if you’re trying to read a summary email on a mobile device, any more than a hundred lines or so will likely be difficult to read and may not even render.
To prevent these problems, SummarisingLogger
allows a limit
on the number of lines that will be included in the summary email.
By default, this is set to 100 messages but can be overridden by passing
the flood_level option to the SummarisingLogger
constructor:
>>> handler = SummarisingLogger('from@example.com', ('to@example.com',),
... flood_level=2)
>>> handler.setFormatter(logging.Formatter('%(levelname)s - %(message)s'))
>>> logger.addHandler(handler)
>>> logging.info('message 1')
>>> logging.info('message 2')
>>> logging.error('message 3')
>>> logging.info('message 4')
>>> logging.info('message 5')
>>> logging.info('message 6')
>>> logging.info('message 7')
>>> logging.info('message 8')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: Mon, 01 Jan 2007 12:34:56 -0000
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
INFO - message 1
INFO - message 2
CRITICAL - 1 messages not included as flood limit of 2 exceeded
INFO - message 4
INFO - message 5
INFO - message 6
INFO - message 7
INFO - message 8
The example above shows a few things. Firstly, when messages are
excluded, a CRITICAL
entry is logged with the number of messages
that have been excluded. Secondly, excluded messages still contribute
to the highest level logged used in both the subject and the
X-Log-Level
header. Finally, the last 5 messages logged before the
mail is sent are always included as they may well contain useful
information such as a terminal exception or total run time.
Sending empty emails
By default, the SummarisingLogger
handler will always send emails
even if they would have been empty:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',))
>>> logger.addHandler(handler)
>>> logging.error(' ')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
Sending empty emails is helpful for batch processes as even if no activity is logged, the mail itself is an indication that the batch process did at least run.
However, if you do not want empty entries to be mailed, all you need to do is supply the send_empty_entries parameter:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... send_empty_entries=False)
>>> logger.addHandler(handler)
>>> logging.error(' ')
>>> logging.shutdown()
Specifying the host to send email through
By default, as we’ve seen above, SummarisingLogger
uses localhost
to send mails. If you wish to use a specific smtp server to send
mail, this can be done by specifying the mailhost parameter to the
SummarisingLogger
constructor:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... mailhost='smtp.example.com')
>>> logger.addHandler(handler)
>>> logging.error('An Error')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('smtp.example.com', 25)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
An Error
If the smtp server you wish to use is running on non-standard port,
you can configure SummarisingLogger
to use this port by specifying
mailhost as a tuple containing the smtp server’s hostname and the
port on which it is listening:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... mailhost=('smtp.example.com',2500))
>>> logger.addHandler(handler)
>>> logging.error('An Error')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('smtp.example.com', 2500)
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
An Error
If the smtp server you wish to use requires authentication,
pass the required username and password to the SummarisingLogger
constructor:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... username='auser',password='theirpassword')
>>> logger.addHandler(handler)
>>> logging.error('An Error')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
(authenticated using username:'auser' and password:'theirpassword')
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
An Error
Warning
For performance reasons, it’s recommended that you don’t use SMTP authentication unless you absolutely need to.
If the smtp server you wish to use requires TLS (Transport Level Security),
pass the required username and password and the secure parameter to the
SummarisingLogger
constructor. secure
must be either a boolean
or an ssl.SSLContext
object:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... username='auser',password='apassword',
... secure=True)
>>> logger.addHandler(handler)
>>> logging.error('An Error')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
(authenticated using username:'auser' and password:'apassword')
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger...
An Error
Adding extra headers
If you wish to add headers for filtering purposes, you can use the headers parameter:
>>> handler = SummarisingLogger('from@example.com',('to@example.com',),
... headers={'foo':'bar','Baz':'bob'})
>>> logger.addHandler(handler)
Now, when a log message results in an email being send, the email will be sent with the configured headers:
>>> logging.error('The Error!')
>>> logging.shutdown()
sending to ('to@example.com',) from 'from@example.com' using ('localhost', 25)
Baz: bob
Content-Transfer-Encoding: 7bit
Content-Type: text/plain; charset="us-ascii"
Date: ...
From: from@example.com
MIME-Version: 1.0
Message-ID: <...MailingLogger@...>
Subject: Summary of Log Messages (ERROR)
To: to@example.com
X-Log-Level: ERROR
X-Mailer: MailingLogger ...
foo: bar
The Error!