t+1 - Latest Comments in The python logging module is much better than print statements

Re: The python logging module is much better than print statements

Teja Ram Seervi — Tue, 24 Feb 2015 05:39:14 -0000

Thanks a lot meow mix.... great explanation.

Re: The python logging module is much better than print statements

Shank — Sun, 20 Jul 2014 09:12:21 -0000

Thank you for this article. It really helps me with switching to logging !

Re: The python logging module is much better than print statements

Matt Wilson — Sun, 27 Jan 2013 20:21:32 -0000

Hey, Thanks for the comment!

In my experience, bugs that crash by throwing uncaught exceptions go away over time as a system spends time in production. After that phase, it becomes more important for a program to leave some kind of audit trail, so you can verify that it is doing what you want it to be doing.

At the same time, if you log every frackin' variable assignment, then you create a needle in the haystack problem.

Two tricks I know of that help here:

1. I use log levels (DEBUG, INFO, etc) to break up what I log and why. I log every state-changing operation as INFO. So, when a user logs in to my web app, that's an INFO. And when my cron script updates some database rows, that's an INFO too.

Every instance of invalid input data gets logged as an ERROR. CRITICAL errors are just the things that I can't recover from, like say, a lost database connection.

I use DEBUG logging for the stuff that I may ned later when debugging. Usually though, the less DEBUG logs, the better. I prefer to log the original state of the program, like maybe the whole HTTP request if I'm doing web programming, and then avoid logging all the various derived values, because I can usually recreate those given that I know the initial inputs.

2. When you make a logger, like::

import logging
log = logging.getLogger('...')

You should use a name that relates to what kind of logging messages you'll put in. In other words, all my database modelling modules have a logger like this::

log = logging.getLogger('project.model')

But I make logs in cron jobs like this:

log = logging.getLogger('project.cron_job')

If you use different named logs like that, you can configure your logging handlers to put stuff in different places. So, for example, you could have all your logs from one part of your program in one text file, separate from another part.

Good luck!

Matt

PS: I did a big ol' talk at PyOhio this summer all about logging. Here's the text from that talk: https://github.com/mw44118/...
There's a lot of "real world" scenarios that I hash through in there. You can also find the video from the talk on youtube. Search for PyOhio really good logging.

Re: The python logging module is much better than print statements

Justin S Barrett — Sat, 26 Jan 2013 11:45:45 -0000

Great article, but I'm still a little confused about how I would apply this in a much larger scenario. I've written a program for a friend's business, and it currently contains thousands of lines of code spanning several modules. Most bugs that throw exceptions are easily found and fixed by the traceback details, but there have been a few times where I've had to add print statements to help me track down problems that *didn't* throw an exception. The program is kicked off by a main() function, so would I simply add logging calls to main(), similar to what you've done with f(), or should I go through the entire thing and add more logging calls in specific places? In other words, how do you determine what level of logging is appropriate for a given situation? I suppose I could log just about everything that happens with the program, but I'd rather not bloat it with so many logging calls that it becomes a huge mess.

Re: The python logging module is much better than print statements

Heather — Sun, 15 Apr 2012 12:42:04 -0000

sometimes, I catch myself reverting to print statements also. It's sooo easy and if the previous developer used it on their coding... I tend to be consistent (if I'm doing their work for them). I have to hear all the wining when a programmer can't read their own code because I didn't modify it according to their ever changing standards. I might as well go back to developing online background checks.
Keep pushing the code, bro...

Re: The python logging module is much better than print statements

Anonymous — Fri, 03 Feb 2012 17:03:50 -0000

Mike Pirnat's slides have been moved to here: http://mike.pirnat.com/stat...

Re: The python logging module is much better than print statements

Matt Wilson — Mon, 05 Dec 2011 20:28:43 -0000

Good luck!

Re: The python logging module is much better than print statements

Eric — Mon, 05 Dec 2011 19:44:39 -0000

Right. Sold. I'm going to use Python Logger as well. Thanks, great article.

Re: The python logging module is much better than print statements

Matt Wilson — Fri, 04 Nov 2011 11:39:21 -0000

grazie!

Re: The python logging module is much better than print statements

Motore Asincrono — Fri, 04 Nov 2011 11:35:58 -0000

great tutorials

Re: The python logging module is much better than print statements

Matt Wilson — Tue, 11 Oct 2011 20:46:22 -0000

Hilarious!

Re: The python logging module is much better than print statements

bahamas10 — Tue, 11 Oct 2011 20:34:38 -0000

thanks meow mix, you really helped me through a tough time of trying not to use print statements.

Re: The python logging module is much better than print statements

cheeseinvert — Thu, 04 Aug 2011 15:12:57 -0000

so true... I often find myself asking the same question when using python logger

Re: The python logging module is much better than print statements

David Noyes — Wed, 23 Mar 2011 21:15:44 -0000

I agree with meow mix

Re: The python logging module is much better than print statements

meow mix coupons — Tue, 08 Mar 2011 11:32:25 -0000

It can be overwhelming and frustrating at times. How do you choose �the one� that works with the �other ones� you will love and care for, and will in return inspire, bring value and joy.
...