Using external modules¶
Since EasyBuild v2.1, support for using modules that were not installed via EasyBuild is available. We refer to such modules as external modules.
This can be useful to reuse vendor-supplied modules, for example on Cray systems.
Using external modules as dependencies¶
External modules can be used as dependencies, by including the module name in the
dependencies list (see
Dependencies), along with the
EXTERNAL_MODULE constant marker.
For example, to specify the readily available module
fftw/220.127.116.11 as a dependency:
dependencies = [('fftw/18.104.22.168', EXTERNAL_MODULE)]
For such dependencies, EasyBuild will:
- load the module before initiating the software build and install procedure
- include a '
module load' statement in the generated module file (for non-build dependencies)
If the specified module is not available, EasyBuild will exit with an error message stating that the dependency can not be resolved because the module could not be found. It will not search for a matching easyconfig file in order to try and install the module to resolve the dependency.
Metadata for external modules¶
Since very little information is available for external modules based on the dependency specification alone (i.e., only the module name), metadata can be supplied to EasyBuild for external modules.
--external-modules-metadata configuration option, the location of one or more files can be specified that
provide such metadata. The files are expected to be in INI format, with a section per module name and key-value
assignments denoting the metadata specific to that module.
[modulename] key1 = value1 key2 = value2, value3
Up until EasyBuild v2.6.0, no metadata for external modules was available by default.
Since EasyBuild v2.7.0, a file providing metadata for Cray-provided modules on Cray systems is included,
and is active by default (i.e., unless other files are specified via
Since EasyBuild v4.1.0, you can use so-called glob patterns to specify a list of paths
--external-modules-metadata, using wildcard characters like
For example, to specify that the metadata for external modules in all
*.cfg files in the directory
should be taken into account, you can use:
For example, the following file provides metadata for a handful of modules that may be provided on Cray systems:
[cray-hdf5/1.8.13] name = HDF5 version = 1.8.13 prefix = HDF5_DIR [cray-hdf5-parallel/1.8.13] name = HDF5 version = 1.8.13 prefix = /opt/cray/hdf5-parallel/1.8.13/GNU/49 [cray-netcdf/4.3.2] name = netCDF, netCDF-Fortran version = 4.3.2, 4.3.2 prefix = NETCDF_DIR [fftw/22.214.171.124] name = FFTW version = 126.96.36.199 prefix = FFTW_INC/..
Supported metadata values¶
The following keys are supported:
name: specifies the software name(s) that is (are) provided by the module
version: specifies the software version(s) that is (are) provided by the module
prefix: specifies the installation prefix of the software that is provided by the module
prefix, a couple of different options are available, i.e.:
- specifying an absolute path (cfr.
cray-hdf5-parallel/1.8.13in the example above)
- specifying the environment variable that is defined by the module and provides the installation prefix
cray-netcdf/4.3.2in the example above)
- this can be optionally combined with a relative path that serves as a 'correction'
fftw/188.8.131.52in the example above) [supported since EasyBuild v2.7.0]
Any other keys are simply ignored.
version are specified, they must provide an equal number of values
(see for example the
cray-netcdf example above).
Handling of provided metadata¶
Using the provided metadata, EasyBuild will define environment variables that are also defined by modules that are generated by EasyBuild itself, if an external module for which metadata is available is loaded as a dependency.
In particular, for each software name that is specified:
- the corresponding environment variable
$EBROOT<NAME>is defined to the specified
prefixvalue (if any)
- the corresponding environment variable
$EBVERSION<NAME>is defined to the corresponding
versionvalue (if any)
For example, for the external modules for which metadata is provided in the example above, the following environment variables are set in the build environment when the module is used as a dependency:
get_software_version functions that are commonly used occasionally in easyblocks
pick up the
$EBVERSION* environment variables, respectively.