Easystack files¶

This documentation covers aspects of specifying a software stack to install with Easybuild with easystack files.

Note: this is an experimental feature. Some of the mentioned functionality may be subject to change or be prone to errors.

Note

Some breaking changes were made to the experimental easystack support in EasyBuild v4.7.0.

Easystack files must now use the easyconfigs top-level key to list easyconfig filenames, as opposed to the software top-level key and separate subkeys like version, versionsuffix and toolchain to specify aspects of an easyconfig file that were supported before.

The basics¶

Easystack files describe an entire software stack, and can be used to specify to EasyBuild what to install.

Usage¶

To build software with Easystack, type:

eb --easystack example.yaml --experimental

where example.yaml is the file with specifications that you just created (more about this in the next section).

Structure of an easystack file¶

Easystack files are written in YAML syntax.

Essentially, an easystack file lists the easyconfig files you want to install, which are specified under the easyconfigs key.

For example:

easyconfigs:
  - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb
  - OpenFOAM-v2206-foss-2022a.eb

Note

You must use '-' to list the easyconfigs in an easystack file.

In addition, you can specify additional configuration options via the options subkey which will only apply to the installation of a particular easyconfig file.

For example:

easyconfigs:
  - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb:
      options:
        from-pr: 15924
        debug: True
  - Hypre-2.25.0-foss-2022a.eb:
  - OpenFOAM-v2206-foss-2022a.eb:
      options:
        installpath: /my/custom/installpath
        parallel: "1"

Note

You need to take care with some values in YAML, especially integers, booleans, etc.

If the specified value definitely must be a string value, you should use quotes ('...') to avoid that the YAML parser automatically converts the value to be of a specific type.

If you wish to use 1 (the number one), such as for the value to parallel option in the example above, then this should be quoted, to avoid it being treated as the boolean True.

In other cases, the automatic conversion is useful, like for the True used above, since debug is a boolean configuration option.

The configuration options that are valid for the eb command can be used (see eb --help), but the - or -- prefixes that are commonly used on the command line are omitted in easystack files.

Using the example easystack file above would be equivalent to running:

eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --from-pr 15924 --debug
eb Hypre-2.25.0-foss-2022a.eb
eb OpenFOAM-v2206-foss-2022a.eb --installpath /my/custom/installpath

Note

Whenever configuration options are not specified (as is the case for Hypre in the example easystack file above), you are still allowed to use ':' after the easyconfig filename: there is no difference in behaviour in ending with or without ':'.

Specifying short options in an easystack file is allowed, for example:

easyconfigs:
  - OpenFOAM-v2206-foss-2022a.eb:
      options:
        D: True

This is not recommended however, as short options are more difficult to interpret by humans.

Combining command line options with options in an easystack file¶

When building software with an easystack file, you can still add additional options on the command line as well. These apply to all items in the easystack file. For example, if you have an easystack file named my_easystack.yaml

easyconfigs:
  - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb:
      options:
        from-pr: 15924
        debug: True
  - OpenFOAM-v2206-foss-2022a.eb:

and you run with

eb --experimental --easystack my_easystack.yaml --dry-run

this will have the same effect as running

eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --dry-run --from-pr 15924 --debug
eb OpenFOAM-v2206-foss-2022a.eb --dry-run --installpath /my/custom/installpath

Note that options specified on the command line are placed before the easyconfig-specific options in the easystack file. EasyBuild will always respect the argument that was put last.

For example:

eb PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb --dry-run --disable-dry-run

will effectively run without enabling dry run mode, since --disable-dry-run is specified after --dry-run.

Since easyconfig-specific configuration options specified in the easystack file are put last, they take priority over the the ones on the command line, if the same configuration option is specified in both.

For example, running:

eb --experimental --easystack my_easystack.yaml --disable-debug

will effectively cause PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb to be installed with debug logging enabled, while OpenFOAM-v2206-foss-2022a.eb will be effectively installed without debug logging.

To be developed¶

In the future, we are planning to support additional also global options specified in the easystack file. For example:

options:
  robot: True
easyconfigs:
  - PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb
  - OpenFOAM-v2206-foss-2022a.eb

would installed both PyTorch-1.12.0-foss-2022a-CUDA-11.7.0.eb and OpenFOAM-v2206-foss-2022a.eb using --robot (see issue #4105).

Additionally, we plan to support specifying for which EasyBuild version an easystack file was intended, which can be helpful in more accurately recreating a certain software stack (see issue #4106).

In the future, the --easystack option will probably be dropped, and EasyBuild will automatically detect the use of easystack files (see issue #4104).

Finally, we plan to support specifying labels, which would make it more easy to install only a certain subset of the items listed in an easystack file. For example, by labelling all GPU-capable software with a gpu label, one could easily choose to not build anything labeled gpu on a CPU node (see issue #3512).