easybuild.framework.easyconfig.tweak module

Easyconfig module that provides functionality for tweaking existing eaysconfig (.eb) files.

author:Stijn De Weirdt (Ghent University)
author:Dries Verdegem (Ghent University)
author:Kenneth Hoste (Ghent University)
author:Pieter De Baets (Ghent University)
author:Jens Timmerman (Ghent University)
author:Toon Willems (Ghent University)
author:Fotis Georgatos (Uni.Lu, NTUA)
author:Alan O’Cais (Juelich Supercomputing Centre)
author:Maxime Boissonneault (Universite Laval, Calcul Quebec, Compute Canada)
easybuild.framework.easyconfig.tweak.check_capability_mapping(source_tc_spec, target_tc_spec)

Compare whether the capabilities of a source toolchain are all present in a target toolchain

Parameters:
  • source_tc_spec – specs of source toolchain
  • target_tc_spec – specs of target toolchain
Returns:

boolean indicating whether or not source toolchain is compatible with target toolchain

easybuild.framework.easyconfig.tweak.ec_filename_for(path)

Return a suiting file name for the easyconfig file at <path>, as determined by its contents.

easybuild.framework.easyconfig.tweak.find_matching_easyconfigs(name, installver, paths)

Find easyconfigs that match specified name/installversion in specified list of paths.

Parameters:
  • name – software name
  • installver – software install version (which includes version, toolchain, versionprefix/suffix, …)
  • paths – list of paths to search easyconfigs in
easybuild.framework.easyconfig.tweak.find_potential_version_mappings(dep, toolchain_mapping, versionsuffix_mapping=None, highest_versions_only=True, ignore_versionsuffixes=False, quiet=False)

Find potential version mapping for a dependency in a new hierarchy

Parameters:
  • dep – dependency specification (dict)
  • toolchain_mapping – toolchain mapping used for search
  • versionsuffix_mapping – mapping of version suffixes (required by software with a special version suffix, such as Python packages)
  • highest_versions_only – only return highest versions
Returns:

list of dependencies that match

easybuild.framework.easyconfig.tweak.get_dep_tree_of_toolchain(toolchain_spec, modtool)

Get list of dependencies of a toolchain (as EasyConfig objects)

Parameters:
  • toolchain_spec – toolchain spec to get the dependencies of
  • modtool – module tool used
Returns:

The dependency tree of the toolchain spec

easybuild.framework.easyconfig.tweak.get_matching_easyconfig_candidates(prefix_stub, toolchain)

Find easyconfigs that match specified requirements w.r.t. toolchain and partial filename pattern.

Parameters:
  • prefix_stub – stub used in regex (e.g., ‘Python-’ or ‘Python-2’)
  • toolchain – the toolchain to use with the search
Returns:

list of candidate paths, toolchain_suffix of candidates

easybuild.framework.easyconfig.tweak.list_deps_versionsuffixes(ec_spec)

Take an easyconfig spec, parse it, extracts the list of version suffixes used in its dependencies

Parameters:ec_spec – location of original easyconfig file
Returns:The list of versionsuffixes used by the dependencies of this recipe
easybuild.framework.easyconfig.tweak.map_common_versionsuffixes(software_name, original_toolchain, toolchain_mapping)

Create a mapping of common versionssuffixes (like -Python-%(pyver)s) between toolchains

Parameters:
  • software_name – Name of software
  • original_toolchain – original toolchain
  • toolchain_mapping – toolchain mapping from that containing original to target
Returns:

dictionary of possible mappings

easybuild.framework.easyconfig.tweak.map_easyconfig_to_target_tc_hierarchy(ec_spec, toolchain_mapping, targetdir=None, update_build_specs=None, update_dep_versions=False, ignore_versionsuffixes=False)

Take an easyconfig spec, parse it, map it to a target toolchain and dump it out

Parameters:
  • ec_spec – location of original easyconfig file
  • toolchain_mapping – mapping between source toolchain and target toolchain
  • targetdir – directory to dump the modified easyconfig file in
  • update_build_specs – dict with names and values of easyconfig parameters to tweak
  • update_dep_versions – boolean indicating whether dependency versions should be updated
Returns:

Location of the modified easyconfig file

easybuild.framework.easyconfig.tweak.map_toolchain_hierarchies(source_toolchain, target_toolchain, modtool)

Create a map between toolchain hierarchy of the initial toolchain and that of the target toolchain

Parameters:
  • source_toolchain – initial toolchain of the easyconfig(s)
  • target_toolchain – target toolchain for tweaked easyconfig(s)
  • modtool – module tool used
Returns:

mapping from source hierarchy to target hierarchy

easybuild.framework.easyconfig.tweak.map_versionsuffixes_cache(func)

Function decorator to cache (and retrieve cached) versionsuffixes mapping between toolchains.

easybuild.framework.easyconfig.tweak.match_minimum_tc_specs(source_tc_spec, target_tc_hierarchy)

Match a source toolchain spec to the minimal corresponding toolchain in a target hierarchy

Parameters:
  • source_tc_spec – specs of source toolchain
  • target_tc_hierarchy – hierarchy of specs for target toolchain
easybuild.framework.easyconfig.tweak.obtain_ec_for(specs, paths, fp=None)

Obtain an easyconfig file to the given specifications.

Either select between available ones, or use the best suited available one to generate a new easyconfig file.

Parameters:
  • specs – list of available easyconfig files
  • paths – a list of paths where easyconfig files can be found
  • fp – the desired file name
easybuild.framework.easyconfig.tweak.pick_version(req_ver, avail_vers)

Pick version based on an optionally desired version and available versions.

If a desired version is specifed, the most recent version that is less recent than or equal to the desired version will be picked; else, the most recent version will be picked.

This function returns both the version to be used, which is equal to the required version if it was specified, and the version picked that matches that closest.

Parameters:
  • req_ver – required version
  • avail_vers – list of available versions
easybuild.framework.easyconfig.tweak.select_or_generate_ec(fp, paths, specs)

Select or generate an easyconfig file with the given requirements, from existing easyconfig files.

If easyconfig files are available for the specified software package, then this function will first try to determine which toolchain to use.

  • if a toolchain is given, it will use it (possible using a template easyconfig file as base);
  • if not, and only a single toolchain is available, is will assume it can use that toolchain
  • else, it fails – EasyBuild doesn’t select between multiple available toolchains

Next, it will trim down the selected easyconfig files to a single one, based on the following requirements (in order of preference):

  • toolchain version
  • software version
  • other parameters (e.g. versionprefix, versionsuffix, etc.)

If a complete match is found, it will return that easyconfig. Else, it will generate a new easyconfig file based on the selected ‘best matching’ easyconfig file.

easybuild.framework.easyconfig.tweak.tweak(easyconfigs, build_specs, modtool, targetdirs=None)

Tweak list of easyconfigs according to provided build specifications.

easybuild.framework.easyconfig.tweak.tweak_one(orig_ec, tweaked_ec, tweaks, targetdir=None)

Tweak an easyconfig file with the given list of tweaks, using replacement via regular expressions. Note: this will only work ‘well-written’ easyconfig files, i.e. ones that e.g. set the version once and then use the ‘version’ variable to construct the list of sources, and possibly other parameters that depend on the version (e.g. list of patch files, dependencies, version suffix, …)

The tweaks should be specified in a dictionary, with parameters and keys that map to the values to be set.

Reads easyconfig file at path <orig_ec>, and writes the tweaked easyconfig file to <tweaked_ec>.

If <tweaked_ec> is not provided, a target filepath is generated based on <targetdir> and the contents of the tweaked easyconfig file.

Parameters:
  • orig_ec – location of original easyconfig file to read
  • tweaked_ec – location where tweaked easyconfig file should be written (if this is None, then filename for tweaked easyconfig is auto-derived from contents)
  • tweaks – dictionary with set of changes to apply to original easyconfig file
  • targetdir – target directory for tweaked easyconfig file, defaults to temporary directory (only used if tweaked_ec is None)