easybuild.base.fancylogger module

This module implements a fancy logger on top of python logging

It adds:

• custom specifiers for mpi logging (the mpirank) with autodetection of mpi
• custom specifier for always showing the calling function’s name
• rotating file handler
• a default formatter.
• logging to an UDP server (vsc.logging.logdaemon.py f.ex.)
• easily setting loglevel
• easily add extra specifiers in the log record

• internal debugging through environment variables

  FANCYLOGGER_GETLOGGER_DEBUG for getLogger FANCYLOGGER_LOGLEVEL_DEBUG for setLogLevel

• set FANCYLOGGER_IGNORE_MPI4PY to disable mpi4py module import

  mpi4py (when available) is automatically used for mpi-aware log format In case mpi4py however that it is available but broken, set this variable to 1 to avoid importing it.

usage:

>>> from easybuild.base import fancylogger
>>> # will log to screen by default
>>> fancylogger.logToFile('dir/filename')
>>> fancylogger.setLogLevelDebug()  # set global loglevel to debug
>>> logger = fancylogger.getLogger(name)  # get a logger with a specific name
>>> logger.setLevel(level)  # set local debugging level
>>> # If you want the logger to be showing modulename.functionname as the name, use
>>> fancylogger.getLogger(fname=True)
>>> # you can use the handler to set a different formatter by using
>>> handler = fancylogger.logToFile('dir/filename')
>>> formatstring = '%(asctime)-15s %(levelname)-10s %(mpirank)-5s %(funcname)-15s %(threadName)-10s %(message)s'
>>> handler.setFormatter(logging.Formatter(formatstring))
>>> # setting a global loglevel will impact all logers:
>>> from easybuild.base import fancylogger
>>> logger = fancylogger.getLogger("test")
>>> logger.warning("warning")
2012-01-05 14:03:18,238 WARNING    <stdin>.test.<module>    MainThread  warning
>>> logger.debug("warning")
>>> fancylogger.setLogLevelDebug()
>>> logger.debug("warning")
2012-01-05 14:03:46,222 DEBUG      <stdin>.test.<module>    MainThread  warning

Logging to a udp server:

• set an environment variable FANCYLOG_SERVER and FANCYLOG_SERVER_PORT (optionally)
• this will make fancylogger log to that that server and port instead of the screen.

author:Jens Timmerman (Ghent University) author:Stijn De Weirdt (Ghent University) author:Kenneth Hoste (Ghent University)

class easybuild.base.fancylogger.FancyLogRecord(*args, **kwargs)

Bases: logging.LogRecord

This class defines a custom log record. Adding extra specifiers is as simple as adding attributes to the log record

class easybuild.base.fancylogger.FancyLogger(name, level=0)

Bases: logging.Logger

This is a custom Logger class that uses the FancyLogRecord and has extra log methods raiseException and deprecated and streaming versions for debug,info,warning and error.

RAISE_EXCEPTION_CLASS

alias of builtins.Exception

RAISE_EXCEPTION_LOG_METHOD(msg)

deprecated(msg, cur_ver, max_ver, depth=2, exception=None, log_callback=None, *args, **kwargs)

Log deprecation message, throw error if current version is passed given threshold.

Checks only major/minor version numbers (MAJ.MIN.x) by default, controlled by ‘depth’ argument.

fail(message, *args)

Log error message and raise exception.

get_parent_info(prefix, verbose=True)

Return pretty text version

log_method(msg)

makeRecord(name, level, pathname, lineno, msg, args, excinfo, func=None, extra=None, sinfo=None)

overwrite make record to use a fancy record (with more options)

raiseException(message, exception=None, catch=False)

logs message and raises an exception (since it can be caught higher up and handled) and raises it afterwards :param exception: subclass of Exception to use for raising :param catch: boolean, try to catch raised exception and add relevant info to message

(this will also happen if exception is not specified)

setLevelName(level_name)

Set the level by name.

streamDebug(data)

Get a DEBUG loglevel streamLog

streamError(data)

Get a ERROR loglevel streamLog

streamInfo(data)

Get a INFO loglevel streamLog

streamLog(levelno, data)

Add (continuous) data to an existing message stream (eg a stream after a logging.info()

class easybuild.base.fancylogger.FancyStreamHandler(stream=None, stdout=None)

Bases: logging.StreamHandler

The logging StreamHandler with uniform named arg in __init__ for selecting the stream.

exception easybuild.base.fancylogger.MissingLevelName

Bases: KeyError

easybuild.base.fancylogger.disableDefaultHandlers()

Disable the default handlers on all fancyloggers - if this is the last logger, it will just set the logLevel very high

easybuild.base.fancylogger.enableDefaultHandlers()

(re)Enable the default handlers on all fancyloggers

easybuild.base.fancylogger.getAllExistingLoggers()

Returns:the existing loggers, in a list of C{(name, logger)} tuples

easybuild.base.fancylogger.getAllFancyloggers()

Return all loggers that are not fancyloggers

easybuild.base.fancylogger.getAllNonFancyloggers()

Returns:all loggers that are not fancyloggers

easybuild.base.fancylogger.getDetailsLogLevels(fancy=True, numeric=False)

Return list of (name,loglevelname) pairs of existing loggers

Parameters:

• fancy – if True, returns only Fancylogger; if False, returns non-FancyLoggers, anything else, return all loggers
• numeric – if True, return the numeric value instead of the name

easybuild.base.fancylogger.getLevelInt(level_name)

Given a level name, return the int value

easybuild.base.fancylogger.getLogger(name=None, fname=False, clsname=False, fancyrecord=None)

Returns a Fancylogger instance if fname is True, the loggers name will be ‘name[.classname].functionname’ if clsname is True the loggers name will be ‘name.classname[.functionname]’

This will return a logger with a fancylog record, which includes the className template for the logformat This can make your code a lot slower, so this can be disabled by setting fancyrecord or class module FANCYLOG_FANCYRECORD to False, or will also be disabled if a Name is set (and fancyrecord and module constant FANCYLOG_FANCYRECORD are also not set).

easybuild.base.fancylogger.getRootLoggerName()

returns the name of the root module this is the module that is actually running everything and so doing the logging

easybuild.base.fancylogger.is_fancyroot()

Return if the logging.root logger is a FancyLogger

easybuild.base.fancylogger.logToDevLog(enable=True, name=None, handler=None)

Log to syslog through /dev/log

easybuild.base.fancylogger.logToFile(filename, enable=True, filehandler=None, name=None, max_bytes=104857600, backup_count=10)

enable (or disable) logging to file given filename will log to a file with the given name using a rotatingfilehandler this will let the file grow to MAX_BYTES and then rotate it saving the last BACKUPCOUNT files.

returns the filehandler (this can be used to later disable logging to file)

if you want to disable logging to file, pass the earlier obtained filehandler

easybuild.base.fancylogger.logToScreen(enable=True, handler=None, name=None, stdout=False, colorize='never')

enable (or disable) logging to screen returns the screenhandler (this can be used to later disable logging to screen)

if you want to disable logging to screen, pass the earlier obtained screenhandler

you can also pass the name of the logger for which to log to the screen otherwise you’ll get all logs on the screen

by default, logToScreen will log to stderr; logging to stdout instead can be done by setting the ‘stdout’ parameter to True

The colorize parameter enables or disables log colorization using ANSI terminal escape sequences, according to the values allowed in the colorize parameter to function _screenLogFormatterFactory (which see).

easybuild.base.fancylogger.logToUDP(hostname, port=5005, enable=True, datagramhandler=None, name=None)

enable (or disable) logging to udp given hostname and port.

returns the filehandler (this can be used to later disable logging to udp)

if you want to disable logging to udp, pass the earlier obtained filehandler, and set boolean = False

easybuild.base.fancylogger.resetroot()

Restore the original logging.root logger

easybuild.base.fancylogger.setLogFormat(f_format)

Set the log format. (Has to be set before logToSomething is called).

easybuild.base.fancylogger.setLogLevel(level)

Set a global log level for all FancyLoggers

easybuild.base.fancylogger.setLogLevelDebug()

shorthand for setting debug level

easybuild.base.fancylogger.setLogLevelError()

shorthand for setting loglevel to Error

easybuild.base.fancylogger.setLogLevelInfo()

shorthand for setting loglevel to Info

easybuild.base.fancylogger.setLogLevelWarning()

shorthand for setting loglevel to Warning

easybuild.base.fancylogger.setTestLogFormat()

Set the log format to the test format (i.e. without timestamp).

easybuild.base.fancylogger.setroot(fancyrecord=None)

Set a FancyLogger instance as the logging root logger with (effective)loglevel of current root FancyLogger

:param fancyrecord is enabled or not (default FANCYLOG_FANCYRECORD module constant)

Detecting the loglevel is best-effort, better to set the loglevel after setroot()

easybuild.base.fancylogger.thread_name()

returns the current threads name