This documentation covers aspects of specifying a software stack to install with Easybuild with easystack files.
Note: this is an experimental feature. Some of the mentioned functionality may be subject to change or be prone to errors.
Some breaking changes were made to the experimental easystack support in EasyBuild v4.7.0.
Easystack files must now use the
easyconfigs top-level key to list easyconfig filenames,
as opposed to the
software top-level key and separate subkeys like
toolchain to specify aspects of an easyconfig file that were supported before.
Easystack files describe an entire software stack, and can be used to specify to EasyBuild what to install.
To build software with Easystack, type:
eb --easystack example.yaml --experimental
example.yaml is the file with specifications that you just
created (more about this in the next section).
Structure of an easystack file¶
Easystack files are written in YAML syntax.
Essentially, an easystack file lists the easyconfig files you want to install, which are specified under the
easyconfigs: - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb - OpenFOAM-v2206-foss-2022a.eb
You must use '
-' to list the easyconfigs in an easystack file.
In addition, you can specify additional configuration options via the
which will only apply to the installation of a particular easyconfig file.
easyconfigs: - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb: options: from-pr: 15924 debug: True - Hypre-2.25.0-foss-2022a.eb: - OpenFOAM-v2206-foss-2022a.eb: options: installpath: /my/custom/installpath parallel: "1"
You need to take care with some values in YAML, especially integers, booleans, etc.
If the specified value definitely must be a string value, you should use quotes (
'...') to avoid
that the YAML parser automatically converts the value to be of a specific type.
If you wish to use 1 (the number one), such as for the value to
parallel option in the example
above, then this should be quoted, to avoid it being treated as the boolean
In other cases, the automatic conversion is useful, like for the
True used above,
debug is a boolean configuration option.
The configuration options that are valid for the
eb command can be used (see
-- prefixes that are commonly used on the command line are omitted in easystack files.
Using the example easystack file above would be equivalent to running:
eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --from-pr 15924 --debug eb Hypre-2.25.0-foss-2022a.eb eb OpenFOAM-v2206-foss-2022a.eb --installpath /my/custom/installpath
Whenever configuration options are not specified (as is the case for
Hypre in the example easystack file above),
you are still allowed to use '
:' after the easyconfig filename: there is no difference in behaviour in ending with or without '
Specifying short options in an easystack file is allowed, for example:
easyconfigs: - OpenFOAM-v2206-foss-2022a.eb: options: D: True
This is not recommended however, as short options are more difficult to interpret by humans.
Combining command line options with options in an easystack file¶
When building software with an easystack file, you can still add additional options on the command line as well.
These apply to all items in the easystack file. For example, if you have an easystack file named
easyconfigs: - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb: options: from-pr: 15924 debug: True - OpenFOAM-v2206-foss-2022a.eb:
and you run with
eb --experimental --easystack my_easystack.yaml --dry-run
this will have the same effect as running
eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --dry-run --from-pr 15924 --debug eb OpenFOAM-v2206-foss-2022a.eb --dry-run --installpath /my/custom/installpath
Note that options specified on the command line are placed before the easyconfig-specific options in the easystack file. EasyBuild will always respect the argument that was put last.
eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --dry-run --disable-dry-run
will effectively run without enabling dry run mode, since
--disable-dry-run is specified after
Since easyconfig-specific configuration options specified in the easystack file are put last, they take priority over the the ones on the command line, if the same configuration option is specified in both.
For example, running:
eb --experimental --easystack my_easystack.yaml --disable-debug
will effectively cause
PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb to be installed with debug logging enabled,
OpenFOAM-v2206-foss-2022a.eb will be effectively installed without debug logging.
To be developed¶
In the future, we are planning to support additional also global options specified in the easystack file. For example:
options: robot: True easyconfigs: - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb - OpenFOAM-v2206-foss-2022a.eb
would installed both
(see issue #4105).
Additionally, we plan to support specifying for which EasyBuild version an easystack file was intended, which can be helpful in more accurately recreating a certain software stack (see issue #4106).
In the future, the
--easystack option will probably be dropped, and EasyBuild will automatically detect
the use of easystack files (see issue #4104).
Finally, we plan to support specifying labels, which would make it more easy to install only a certain subset of
the items listed in an easystack file. For example, by labelling all GPU-capable software with a
one could easily choose to not build anything labeled
gpu on a CPU node
(see issue #3512).