easybuild.base.generaloption module

A class that can be used to generated options to python scripts in a general way.

author:Stijn De Weirdt (Ghent University)
author:Jens Timmerman (Ghent University)
class easybuild.base.generaloption.ExtOption(*args, **kwargs)

Bases: easybuild.base.optcomplete.CompleterOption

Extended options class
  • enable/disable support
Actions:
  • shorthelp : hook for shortend help messages
  • confighelp : hook for configfile-style help messages
  • store_debuglog : turns on fancylogger debugloglevel
    • also: ‘store_infolog’, ‘store_warninglog’
  • add : add value to default (result is default + value)
    • add_first : add default to value (result is value + default)
    • extend : alias for add with strlist type
    • type must support + (__add__) and one of negate (__neg__) or slicing (__getslice__)
  • add_flex : similar to add / add_first, but replaces the first “empty” element with the default
    • the empty element is dependent of the type
      • for {str,path}{list,tuple} this is the empty string
    • types must support the index method to determine the location of the “empty” element
    • the replacement uses +
    • e.g. a strlist type with value “0,,1”` and default [3,4] and action add_flex will
      use the empty string ‘’ as “empty” element, and will result in [0,3,4,1] (not [0,[3,4],1]) (but also a strlist with value “” and default [3,4] will result in [3,4]; so you can’t set an empty list with add_flex)
  • date : convert into datetime.date
  • datetime : convert into datetime.datetime
  • regex: compile str in regexp
  • store_or_None - set default to None if no option passed, - set to default if option without value passed, - set to value if option with value passed
Types:
  • strlist, strtuple : convert comma-separated string in a list resp. tuple of strings
  • pathlist, pathtuple : using os.pathsep, convert pathsep-separated string in a list resp. tuple of strings
    • the path separator is OS-dependent
ACTIONS = ('store', 'store_const', 'store_true', 'store_false', 'append', 'append_const', 'count', 'callback', 'help', 'version', 'date', 'datetime', 'regex', 'add', 'add_first', 'add_flex', 'store_or_None', 'help', 'store_debuglog', 'store_infolog', 'store_warninglog', 'shorthelp', 'confighelp', 'help')
ALWAYS_TYPED_ACTIONS = ('store', 'append', 'date', 'datetime', 'regex', 'add', 'add_first', 'add_flex')
BOOLEAN_ACTIONS = ('store_true', 'store_false', 'store_debuglog', 'store_infolog', 'store_warninglog')
DISABLE = 'disable'
ENABLE = 'enable'
EXTEND_SEPARATOR = ','
EXTOPTION_EXTRA_OPTIONS = ('date', 'datetime', 'regex', 'add', 'add_first', 'add_flex')
EXTOPTION_HELP = ('shorthelp', 'confighelp', 'help')
EXTOPTION_LOG = ('store_debuglog', 'store_infolog', 'store_warninglog')
EXTOPTION_STORE_OR = ('store_or_None', 'help')
STORE_ACTIONS = ('store', 'store_const', 'store_true', 'store_false', 'append', 'append_const', 'count', 'date', 'datetime', 'regex', 'add', 'add_first', 'add_flex', 'store_debuglog', 'store_infolog', 'store_warninglog', 'store_or_None')
TYPED_ACTIONS = ('store', 'append', 'callback', 'date', 'datetime', 'regex', 'add', 'add_first', 'add_flex', 'store_or_None', 'help')
TYPES = ('strlist', 'pathlist', 'strtuple', 'pathtuple', 'string', 'int', 'long', 'float', 'complex', 'choice')
TYPE_CHECKER = {'choice': <function check_choice>, 'complex': <function check_builtin>, 'float': <function check_builtin>, 'int': <function check_builtin>, 'long': <function check_builtin>, 'pathlist': <function check_str_list_tuple>, 'pathtuple': <function check_str_list_tuple>, 'strlist': <function check_str_list_tuple>, 'strtuple': <function check_str_list_tuple>}
TYPE_STRLIST = ['strlist', 'pathlist', 'strtuple', 'pathtuple']
process(opt, value, values, parser)

Handle option-as-value issues before actually processing option.

take_action(action, dest, opt, value, values, parser)

Extended take_action

class easybuild.base.generaloption.ExtOptionGroup(*args, **kwargs)

Bases: optparse.OptionGroup

An OptionGroup with support for configfile section names

NO_SECTION = ('NO', 'SECTION')
RESERVED_SECTIONS = ['DEFAULT']
add_option(*args, **kwargs)

Extract configfile section info

class easybuild.base.generaloption.ExtOptionParser(*args, **kwargs)

Bases: optparse.OptionParser

Make an option parser that limits the C{-h} / C{–shorthelp} to short opts only, C{-H} / C{–help} for all options.

Pass options through environment. Like:

  • C{export PROGNAME_SOMEOPTION = value} will generate {–someoption=value}
  • C{export PROGNAME_OTHEROPTION = 1} will generate {–otheroption}
  • C{export PROGNAME_OTHEROPTION = 0} (or no or false) won’t do anything

distinction is made based on option.action in TYPED_ACTIONS allow C{–enable-} / C{–disable-} (using eg ExtOption option_class)

ALLOW_DASH_AS_VALUE = False
ALLOW_OPTION_AS_VALUE = False
ALLOW_OPTION_NAME_AS_VALUE = False
ALLOW_TYPO_AS_VALUE = True
DESCRIPTION_DOCSTRING = False
VALUES_CLASS

alias of optparse.Values

check_help(fh)

Checks filehandle for help functions

format_description(formatter)

Extend to allow docstring as description

format_epilog(formatter)

Allow multiple epilog parts

format_help(formatter=None)

For py2.4 compatibility reasons (missing epilog). This is the py2.7 / optparse 1.5.3 code

format_option_rsthelp(formatter=None)

Formatting for help in rst format

get_default_values()

Introduce the ExtValues class with class constant - make it dynamic, otherwise the class constant is shared between multiple instances - class constant is used to avoid _action_taken as option in the __dict__

  • only works by using reference to object
  • same for _logaction_taken
get_env_options()

Retrieve options from the environment: prefix_longopt.upper()

get_env_options_prefix()

Return the prefix to use for options passed through the environment

get_option_by_long_name(name)

Return the option matching the long option name

is_value_a_commandline_option(opt, value, index=None)

Determine if value is/could be an option passed via the commandline. If it is, return the reason why (can be used as message); or return None if it isn’t.

opt is the option flag to which the value is passed; index is the index of the value on the commandline (if None, it is determined from orig_rargs and rargs)

The method tests for possible ambiguity on the commandline when the parser interprets the argument following an option as a value, whereas it is far more likely that it is (intended as) an option; –longopt=value is never considered ambiguous, regardless of the value.

longhelp = ('H', '--help')
print_confighelp(fh=None)

Print help as a configfile.

print_help(fh=None)

Intercept print to file to print to string and remove the ENABLE/DISABLE options from help

print_rsthelp(fh=None)

Print help in rst format

print_shorthelp(fh=None)

Print a shortened help (no longopts)

set_description_docstring()

Try to find the main docstring and add it if description is not None

set_usage(usage)

Return usage and set try to set autogenerated description.

shorthelp = ('h', '--shorthelp')
class easybuild.base.generaloption.GeneralOption(**kwargs)

Bases: object

‘Used-to-be simple’ wrapper class for option parsing

Options with go_ prefix are for this class, the remainder is passed to the parser
  • go_args : use these instead of of sys.argv[1:]
  • go_columns : specify column width (in columns)
  • go_useconfigfiles : use configfiles or not (default set by CONFIGFILES_USE)
    if True, an option –configfiles will be added
  • go_configfiles : list of configfiles to parse. Uses ConfigParser.read; last file wins
  • go_configfiles_initenv : section dict of key/value dict; inserted before configfileparsing
    As a special case, using all uppercase key in DEFAULT section with a case-sensitive configparser can be used to set “constants” for easy interpolation in all sections.
  • go_loggername : name of logger, default classname
  • go_mainbeforedefault : set the main options before the default ones
  • go_autocompleter : dict with named options to pass to the autocomplete call (eg arg_completer)
    if is None: disable autocompletion; default is {} (ie no extra args passed)

Sections starting with the string ‘raw_’ in the sectionname will be parsed as raw sections, meaning there will be no interpolation of the strings. This comes in handy if you want to configure strings with templates in them.

Options process order (last one wins)
  1. default defined with option
  2. value in (last) configfile (last configfile wins)

2. options parsed by option parser In case the ExtOptionParser is used

  1. value set through environment variable
  2. value set through commandline option
ALLOPTSMANDATORY = True
CONFIGFILES_IGNORE = []
CONFIGFILES_INIT = []
CONFIGFILES_MAIN_SECTION = 'MAIN'
CONFIGFILES_RAISE_MISSING = False
CONFIGFILES_USE = True
CONFIGFILE_CASESENSITIVE = True
CONFIGFILE_PARSER

alias of configparser.SafeConfigParser

DEBUG_OPTIONS_BUILD = False
DEFAULTSECT = 'DEFAULT'
DEFAULT_CONFIGFILES = None
DEFAULT_IGNORECONFIGFILES = None
DEFAULT_LOGLEVEL = None
INTERSPERSED = True
METAVAR_DEFAULT = True
METAVAR_MAP = None
OPTIONGROUP_SORTED_OPTIONS = True
OPTIONNAME_PREFIX_SEPARATOR = '-'
PARSER

alias of ExtOptionParser

PROCESSED_OPTIONS_PROPERTIES = ['type', 'default', 'action', 'opt_name', 'prefix', 'section_name']
SETROOTLOGGER = False
USAGE = None
VERSION = None
add_group_parser(opt_dict, description, prefix=None, otherdefaults=None, section_name=None)

Make a group parser from a dict

@type opt_dict: dict @type description: a 2 element list (short and long description) @section_name: str, the name of the section group in the config file.

Parameters:opt_dict – options, with the form C{“long_opt” : value}.

Value is a C{tuple} containing C{(help,type,action,default(,optional string=short option; list/tuple=choices; dict=add_option kwargs))}

help message passed through opt_dict will be extended with type and default

If section_name is None, prefix will be used. If prefix is None or ‘’, ‘DEFAULT’ is used.

autocomplete()

Set the autocompletion magic via optcomplete

configfile_parser_init(initenv=None)

Initialise the configparser to use.

params initenv:insert initial environment into the configparser. It is a dict of dicts; the first level key is the section name; the 2nd level key,value is the key=value. All section names, keys and values are converted to strings.
default_parseoptions()

Return default options

dict_by_prefix(merge_empty_prefix=False)

Break the options dict by prefix; return nested dict. :param merge_empty_prefix : boolean (default False) also (try to) merge the empty

prefix in the root of the dict. If there is a non-prefixed optionname that matches a prefix, it will be rejected and error will be logged.
generate_cmd_line(ignore=None, add_default=None)

Create the commandline options that would create the current self.options. The result is sorted on the destination names.

:param ignore : regex on destination :param add_default : print value that are equal to default
get_options_by_prefix(prefix)

Get all options that set with prefix. Return a dict. The keys are stripped of the prefix.

get_options_by_section(section)

Get all options from section. Return a dict.

main_options()

Create the main options automatically

make_option_metavar(longopt, details)

Generate the metavar for option longopt @type longopt: str @type details: tuple

make_options_option_name_and_destination(prefix, key)

Make the options option name

parseconfigfiles()

Parse configfiles

parseoptions(options_list=None)

Parse the options

postprocess()

Some additional processing

set_go_debug()

Check if debug options are on and then set fancylogger to debug. This is not the default way to set debug, it enables debug logging in an earlier stage to debug generaloption itself.

validate()

Final step, allows for validating the options and/or args

class easybuild.base.generaloption.SimpleOption(go_dict=None, descr=None, short_groupdescr=None, long_groupdescr=None, config_files=None)

Bases: easybuild.base.generaloption.GeneralOption

PARSER

alias of SimpleOptionParser

SETROOTLOGGER = True
main_options()

Create the main options automatically

class easybuild.base.generaloption.SimpleOptionParser(*args, **kwargs)

Bases: easybuild.base.generaloption.ExtOptionParser

DESCRIPTION_DOCSTRING = True
easybuild.base.generaloption.check_str_list_tuple(option, opt, value)
check function for strlist and strtuple type
assumes value is comma-separated list returns list or tuple of strings
easybuild.base.generaloption.get_empty_add_flex(allvalues, self=None)

Return the empty element for add_flex action for allvalues

easybuild.base.generaloption.get_translation()
easybuild.base.generaloption.set_columns(cols=None)

Set os.environ COLUMNS variable - only if it is not set already

easybuild.base.generaloption.simple_option(go_dict=None, descr=None, short_groupdescr=None, long_groupdescr=None, config_files=None)

A function that returns a single level GeneralOption option parser

:param go_dict : General Option option dict :param short_descr : short description of main options :param long_descr : longer description of main options :param config_files : list of configfiles to read options from

a general options dict has as key the long option name, and is followed by a list/tuple mandatory are 4 elements : option help, type, action, default a 5th element is optional and is the short help name (if any)

the generated help will include the docstring

easybuild.base.generaloption.what_str_list_tuple(name)

Given name, return separator, class and helptext wrt separator. (Currently supports strlist, strtuple, pathlist, pathtuple)