easybuild.tools.run module

Tools to run commands.

author:Stijn De Weirdt (Ghent University) author:Dries Verdegem (Ghent University) author:Kenneth Hoste (Ghent University) author:Pieter De Baets (Ghent University) author:Jens Timmerman (Ghent University) author:Toon Willems (Ghent University) author:Ward Poelmans (Ghent University)

easybuild.tools.run.check_async_cmd(proc, cmd, owd, start_time, cmd_log, fail_on_error=True, output_read_size=1024, output='')

Check status of command that was started asynchronously.

Parameters:

• proc – subprocess.Popen instance representing asynchronous command
• cmd – command being run
• owd – original working directory
• start_time – start time of command (datetime instance)
• cmd_log – log file to print command output to
• fail_on_error – raise EasyBuildError when command exited with an error
• output_read_size – number of bytes to read from output
• output – already collected output for this command

Result:

dict value with result of the check (boolean ‘done’, ‘exit_code’, ‘output’)

easybuild.tools.run.check_log_for_errors(log_txt, reg_exps)

Check log_txt for messages matching regExps in order and do appropriate action :param log_txt: String containing the log, will be split into individual lines :param reg_exps: List of: regular expressions (as strings) to error on,

or tuple of regular expression and action (any of [IGNORE, WARN, ERROR])

easybuild.tools.run.complete_cmd(proc, cmd, owd, start_time, cmd_log, log_ok=True, log_all=False, simple=False, regexp=True, stream_output=None, trace=True, output='')

Complete running of command represented by passed subprocess.Popen instance.

Parameters:

• proc – subprocess.Popen instance representing running command
• cmd – command being run
• owd – original working directory
• start_time – start time of command (datetime instance)
• cmd_log – log file to print command output to
• log_ok – only run output/exit code for failing commands (exit code non-zero)
• log_all – always log command output and exit code
• simple – if True, just return True/False to indicate success, else return a tuple: (output, exit_code)
• regex – regex used to check the output for errors; if True it will use the default (see parse_log_for_error)
• stream_output – enable streaming command output to stdout
• trace – print command being executed as part of trace output

easybuild.tools.run.extract_errors_from_log(log_txt, reg_exps)

Check provided string (command output) for messages matching specified regular expressions, and return 2-tuple with list of warnings and errors. :param log_txt: String containing the log, will be split into individual lines :param reg_exps: List of: regular expressions (as strings) to error on,

or tuple of regular expression and action (any of [IGNORE, WARN, ERROR])

:return (warnings, errors) as lists of lines containing a match

easybuild.tools.run.get_output_from_process(proc, read_size=None, asynchronous=False)

Get output from running process (that was opened with subprocess.Popen).

Parameters:

• proc – process to get output from
• read_size – number of bytes of output to read (if None: read all output)
• asynchronous – get output asynchronously

easybuild.tools.run.parse_cmd_output(cmd, stdouterr, ec, simple, log_all, log_ok, regexp)

Parse command output and construct return value. :param cmd: executed command :param stdouterr: combined stdout/stderr of executed command :param ec: exit code of executed command :param simple: if True, just return True/False to indicate success, else return a tuple: (output, exit_code) :param log_all: always log command output and exit code :param log_ok: only run output/exit code for failing commands (exit code non-zero) :param regex: regex used to check the output for errors; if True it will use the default (see parse_log_for_error)

easybuild.tools.run.parse_log_for_error(txt, regExp=None, stdout=True, msg=None)

txt is multiline string. - in memory regExp is a one-line regular expression - default

easybuild.tools.run.run_cmd(cmd, log_ok=True, log_all=False, simple=False, inp=None, regexp=True, log_output=False, path=None, force_in_dry_run=False, verbose=True, shell=None, trace=True, stream_output=None, asynchronous=False)

Run specified command (in a subshell) :param cmd: command to run :param log_ok: only run output/exit code for failing commands (exit code non-zero) :param log_all: always log command output and exit code :param simple: if True, just return True/False to indicate success, else return a tuple: (output, exit_code) :param inp: the input given to the command via stdin :param regex: regex used to check the output for errors; if True it will use the default (see parse_log_for_error) :param log_output: indicate whether all output of command should be logged to a separate temporary logfile :param path: path to execute the command in; current working directory is used if unspecified :param force_in_dry_run: force running the command during dry run :param verbose: include message on running the command in dry run output :param shell: allow commands to not run in a shell (especially useful for cmd lists), defaults to True :param trace: print command being executed as part of trace output :param stream_output: enable streaming command output to stdout :param asynchronous: run command asynchronously (returns subprocess.Popen instance if set to True)

easybuild.tools.run.run_cmd_cache(func)

Function decorator to cache (and retrieve cached) results of running commands.

easybuild.tools.run.run_cmd_qa(cmd, qa, no_qa=None, log_ok=True, log_all=False, simple=False, regexp=True, std_qa=None, path=None, maxhits=50, trace=True)

Run specified interactive command (in a subshell) :param cmd: command to run :param qa: dictionary which maps question to answers :param no_qa: list of patters that are not questions :param log_ok: only run output/exit code for failing commands (exit code non-zero) :param log_all: always log command output and exit code :param simple: if True, just return True/False to indicate success, else return a tuple: (output, exit_code) :param regex: regex used to check the output for errors; if True it will use the default (see parse_log_for_error) :param std_qa: dictionary which maps question regex patterns to answers :param path: path to execute the command is; current working directory is used if unspecified :param maxhits: maximum number of cycles (seconds) without being able to find a known question :param trace: print command being executed as part of trace output