gc3pie
Interface for submitting jobs via GC3Pie.
Authors:
- Riccardo Murri (University of Zurich)
- Kenneth Hoste (Ghent University)
AbortingDependentTaskCollection
¶
Bases: AbortOnError, DependentTaskCollection
A DependentTaskCollection:class: that aborts execution upon error.
This is used to stop the build process in case some dependency
fails. See also <https://github.com/easybuilders/easybuild-framework/issues/1441>_
GC3Pie
¶
Bases: JobBackend
Use the GC3Pie framework to submit and monitor compilation jobs, see http://gc3pie.readthedocs.org/.
In contrast with accessing an external service, GC3Pie implements
its own workflow manager, which means eb --job
--job-backend=GC3Pie will keep running until all jobs have
terminated.
__init__(*args, **kwargs)
¶
GC3Pie JobBackend constructor.
complete()
¶
Complete a bulk job submission.
Create engine, and progress it until all jobs have terminated.
init()
¶
Initialise the GC3Pie job backend.
make_job(script, name, env_vars=None, hours=None, cores=None)
¶
Create and return a job object with the given parameters.
Argument script is the content of the job script itself, i.e., the sequence of shell commands that will be executed.
Argument name sets the job's human-readable name.
Optional argument env_vars is a dictionary with key-value pairs of environment variables that should be passed on to the job.
Optional arguments hours and cores should be
integer values:
- hours must be in the range 1 .. MAX_WALLTIME;
- cores depends on which cluster the job is being run.
queue(job, dependencies=frozenset())
¶
Add a job to the queue, optionally specifying dependencies.
| PARAMETER | DESCRIPTION |
|---|---|
dependencies |
jobs on which this job depends.
DEFAULT:
|