Overview of changes in EasyBuild version 4.0¶
This page provides a concise overview of the most prominent changes in EasyBuild version 4.0, compared to the latest EasyBuild v3.x (version 3.9.4).
Significant enhancements in EasyBuild v4.0¶
Various significant enhancements are included in EasyBuild v4.0, including:
- No more required Python packages
- Support for running EasyBuild on top of Python 3
- Custom software-specific easyblocks for
- 2019b update of common toolchains
- Local variables in easyconfigs should follow recommended naming scheme
No more required Python packages¶
To simplify the installation of EasyBuild, no Python packages other than the ones included in the Python standard library are required for EasyBuild v4.0. More specifically:
setuptoolsis no longer a required dependency, neither for using EasyBuild nor for installing it. This change was motivated by the various problems with installing EasyBuild that were reported, which could often be attributed to
setuptoolsin one way or another (ancient versions being installed via the OS package manager, quirky or broken functionality in specific versions, etc.).
vsc-installare no longer required dependencies. The relevant parts of these packages were ingested into the EasyBuild framework codebase itself (see also Ingested functionality from
vsc-install), mostly to facilitate making EasyBuild compatible with Python 3.
Note that specific Python packages may currently still be required for certain EasyBuild functionality outside
of the core functionality of building and installing scientific software, including
keyring for the GitHub
integration features (see Integration with GitHub), etc.
Support for running EasyBuild on top of Python 3¶
A significant effort has been made to ensure that the EasyBuild framework and the easyblocks included with EasyBuild are compatible with Python 3, while retaining the compatibility with Python 2.
Currently supported Python versions include:
To achieve this in a maintainable way, the
easybuild.tools.py2vs3 package was introduced.
For more details, please see Compatibility with Python 2 and Python 3.
Custom software-specific easyblocks for
A couple of additional custom software-specific easyblocks were added in EasyBuild v4.0, including:
iccifort: to install
iforttogether in a single prefix, and use that installation as a toolchain
numexpr: to ensure that
numexpris linked with Intel MKL's VML when it is available
OpenMPI: to improve the (default) configuration of
OpenMPIbased on installed OS packages (for example for InfiniBand support) and available dependencies
In addition, various (generic and software-specific) easyblocks were improved and enhanced (see EasyBuild release notes for more details).
2019b update of common toolchains¶
2019b update of the common toolchains is included with EasyBuild v4.0; see also Common toolchains.
Local variables in easyconfigs should follow recommended naming scheme¶
A recommended naming scheme for local variables in easyconfig files was introduced.
If an easyconfig file contains local variables that do not follow the recommended naming scheme, a warning will be printed (by default).
For more information, see Local variables in easyconfig files.
Backwards-incompatible changes in EasyBuild v4.0¶
A couple of backwards-incompatible changes were made in EasyBuild v4.0:
--fixed-installdir-naming-schemeenabled by default
- Relocated functions, classes and constants in EasyBuild framework
- Ingested functionality from
- Functions & classes that must be imported from the
Other than the changed default configuration, these changes are mainly important for developers of the EasyBuild framework and easyblock implementers.
--fixed-installdir-naming-scheme enabled by default¶
--fixed-installdir-naming-scheme was changed to be enabled by default,
which makes the name of the software installation directory independent of the module naming scheme being used.
With this configuration setting enabled, the name of software installation directories will match the module name
generated by the (default)
EasyBuildMNS module naming scheme.
So, for an easyconfig file
example-1.2.3-foss-2019b.eb, the software installation directory will always be
<prefix>/software/example/1.2.3-foss-2019b, regardless of the active module naming scheme.
This change was done to make it easier to generate module files using one or more additional module naming scheme
--module-only) for already existing software installations. Only when the name of the software installation
directories is independent of the module naming scheme being used during the (first) installation is it possible to
later generate additional module files using a different module naming scheme.
--fixed-installdir-naming-scheme enabled should not cause problems in existing setups.
Mixing different naming schemes for software installations directories in a single software stack works just fine; it's mostly a matter of inconsistency that could be confusing for humans.
If you are using a module naming scheme other than the default
EasyBuildMNS, you may prefer disabling
--fixed-installdir-naming-scheme to maintain consistency in the names of software installation directories.
Do note that this implies that you won't be able to generate additional modules files using a different module
naming scheme for existing installations.
--fixed-installdir-naming-scheme, you can either
fixed-installdir-naming-schemeconfiguration option in the
[override]section of an EasyBuild configuration file:
[override] fixed-installdir-naming-scheme = 0
(see also Configuration file(s))
(see also Environment variables)
--disable-fixed-installdir-naming-scheme ...command line option:
eb --disable-fixed-installdir-naming-scheme ...
(see also Command line arguments)
Relocated functions, classes and constants in EasyBuild framework¶
A limited number of functions, classes and constants have been relocated in the EasyBuild framework.
This was done mostly to ensure that the
__init__.py files that define packages are empty, other
pkgutil.extend_path (which replaces the
pkg_resources.declare_namespace which requires
see also No more required Python packages).
Making EasyBuild compatible with Python 3 also required some similar changes, see Functions & classes that must be imported from the
A detailed overview of relocated functions and constants is available at Overview of relocated functions, classes and constants in EasyBuild v4.0.
Ingested functionality from
The functionality from the
vsc-install packages required by EasyBuild has been ingested
in the EasyBuild framework, see also No more required Python packages.
This has primarily been done in the new
easybuild.base package, so in general imports
from a module in the
vsc.utils package should be replaced with a corresponding import statement
from that same module in the
A number of specific functions and classes have been placed in existing modules in the
easybuild.tools package instead.
The affected functions are listed in Overview of relocated functions, classes and constants in EasyBuild v4.0.
To avoid that any functionality is imported from an already installed
which could lead to strange side-effects, a fake
vsc namespace is injected since EasyBuild v4.0.
If an import from
vsc.utils.* is detected (for example from an easyblock or from a module that is included
via one of the
--include-* options), an error like this will be produced:
ERROR: Detected import from 'vsc' namespace in /home/example/old_easyblock.py (line 7) vsc-base & vsc-install were ingested into the EasyBuild framework in EasyBuild v4.0 The functionality you need may be available in the 'easybuild.base.*' namespace.
Functions & classes that must be imported from the
A handful functions and classes that could be imported directly from the Python standard library should now be
imported from the
easybuild.tools.py2vs3 namespace instead (perhaps under a different name),
to ensure compatibility with Python 2 and 3.
See Compatibility with Python 2 and Python 3 for more information.
The affected functions are also included in Overview of relocated functions, classes and constants in EasyBuild v4.0.
Deprecated functionality in EasyBuild v4.0¶
Some functionality was deprecated in EasyBuild v4.0, and will no longer be supported in EasyBuild v5.0.
If you trigger any deprecated functionality, a warning message will be printed.
dummy toolchain is now deprecated, and has been replaced with the
For more information, please consult System toolchain.