run

Tools to run commands.

Authors:

• Stijn De Weirdt (Ghent University)
• Dries Verdegem (Ghent University)
• Kenneth Hoste (Ghent University)
• Pieter De Baets (Ghent University)
• Jens Timmerman (Ghent University)
• Toon Willems (Ghent University)
• Ward Poelmans (Ghent University)

check_async_cmd(proc, cmd, owd, start_time, cmd_log, fail_on_error=True, output_read_size=1024, output='')

Check status of command that was started asynchronously.

:result: dict value with result of the check (boolean 'done', 'exit_code', 'output')

PARAMETER	DESCRIPTION
proc	subprocess.Popen instance representing asynchronous command
cmd	command being run
owd	original working directory
start_time	start time of command (datetime instance)
cmd_log	log file to print command output to
fail_on_error	raise EasyBuildError when command exited with an error DEFAULT: True
output_read_size	number of bytes to read from output DEFAULT: 1024
output	already collected output for this command DEFAULT: ''

check_log_for_errors(log_txt, reg_exps)

Check log_txt for messages matching regExps in order and do appropriate action

PARAMETER	DESCRIPTION
log_txt	String containing the log, will be split into individual lines
reg_exps	List of: regular expressions (as strings) to error on, or tuple of regular expression and action (any of [IGNORE, WARN, ERROR])

complete_cmd(proc, cmd, owd, start_time, cmd_log, log_ok=True, log_all=False, simple=False, regexp=True, stream_output=None, trace=True, output='', with_hook=True)

Complete running of command represented by passed subprocess.Popen instance.

PARAMETER	DESCRIPTION
proc	subprocess.Popen instance representing running command
cmd	command being run
owd	original working directory
start_time	start time of command (datetime instance)
cmd_log	log file to print command output to
log_ok	only run output/exit code for failing commands (exit code non-zero) DEFAULT: True
log_all	always log command output and exit code DEFAULT: False
simple	if True, just return True/False to indicate success, else return a tuple: (output, exit_code) DEFAULT: False
regexp	regex used to check the output for errors; if True it will use the default (see parse_log_for_error) DEFAULT: True
stream_output	enable streaming command output to stdout DEFAULT: None
trace	print command being executed as part of trace output DEFAULT: True
with_hook	trigger post run_shell_cmd hooks (if defined) DEFAULT: True

extract_errors_from_log(log_txt, reg_exps)

Check provided string (command output) for messages matching specified regular expressions, and return 2-tuple with list of warnings and errors.

PARAMETER	DESCRIPTION
log_txt	String containing the log, will be split into individual lines
reg_exps	List of: regular expressions (as strings) to error on, or tuple of regular expression and action (any of [IGNORE, WARN, ERROR])

RETURNS	DESCRIPTION
	(warnings, errors) as lists of lines containing a match

get_output_from_process(proc, read_size=None, asynchronous=False)

Get output from running process (that was opened with subprocess.Popen).

PARAMETER	DESCRIPTION
proc	process to get output from
read_size	number of bytes of output to read (if None: read all output) DEFAULT: None
asynchronous	get output asynchronously DEFAULT: False

parse_cmd_output(cmd, stdouterr, ec, simple, log_all, log_ok, regexp)

Parse command output and construct return value.

PARAMETER	DESCRIPTION
cmd	executed command
stdouterr	combined stdout/stderr of executed command
ec	exit code of executed command
simple	if True, just return True/False to indicate success, else return a tuple: (output, exit_code)
log_all	always log command output and exit code
log_ok	only run output/exit code for failing commands (exit code non-zero)
regexp	regex used to check the output for errors; if True it will use the default (see parse_log_for_error)

parse_log_for_error(txt, regExp=None, stdout=True, msg=None)

txt is multiline string. - in memory regExp is a one-line regular expression - default

run_cmd(cmd, log_ok=True, log_all=False, simple=False, inp=None, regexp=True, log_output=False, path=None, force_in_dry_run=False, verbose=True, shell=None, trace=True, stream_output=None, asynchronous=False, with_hooks=True)

Run specified command (in a subshell)

PARAMETER	DESCRIPTION
cmd	command to run
log_ok	only run output/exit code for failing commands (exit code non-zero) DEFAULT: True
log_all	always log command output and exit code DEFAULT: False
simple	if True, just return True/False to indicate success, else return a tuple: (output, exit_code) DEFAULT: False
inp	the input given to the command via stdin DEFAULT: None
regexp	regex used to check the output for errors; if True it will use the default (see parse_log_for_error) DEFAULT: True
log_output	indicate whether all output of command should be logged to a separate temporary logfile DEFAULT: False
path	path to execute the command in; current working directory is used if unspecified DEFAULT: None
force_in_dry_run	force running the command during dry run DEFAULT: False
verbose	include message on running the command in dry run output DEFAULT: True
shell	allow commands to not run in a shell (especially useful for cmd lists), defaults to True DEFAULT: None
trace	print command being executed as part of trace output DEFAULT: True
stream_output	enable streaming command output to stdout DEFAULT: None
asynchronous	run command asynchronously (returns subprocess.Popen instance if set to True) DEFAULT: False
with_hooks	trigger pre/post run_shell_cmd hooks (if defined) DEFAULT: True

run_cmd_cache(func)

Function decorator to cache (and retrieve cached) results of running commands.

run_cmd_qa(cmd, qa, no_qa=None, log_ok=True, log_all=False, simple=False, regexp=True, std_qa=None, path=None, maxhits=50, trace=True)

Run specified interactive command (in a subshell)

PARAMETER	DESCRIPTION
cmd	command to run
qa	dictionary which maps question to answers
no_qa	list of patters that are not questions DEFAULT: None
log_ok	only run output/exit code for failing commands (exit code non-zero) DEFAULT: True
log_all	always log command output and exit code DEFAULT: False
simple	if True, just return True/False to indicate success, else return a tuple: (output, exit_code) DEFAULT: False
regexp	regex used to check the output for errors; if True it will use the default (see parse_log_for_error) DEFAULT: True
std_qa	dictionary which maps question regex patterns to answers DEFAULT: None
path	path to execute the command is; current working directory is used if unspecified DEFAULT: None
maxhits	maximum number of cycles (seconds) without being able to find a known question DEFAULT: 50
trace	print command being executed as part of trace output DEFAULT: True