.. _configuring_easybuild: Configuring EasyBuild ===================== This page discusses the recommended style of configuring EasyBuild, which is supported since EasyBuild v1.3.0. A demo on configuring EasyBuild is available :ref:`here `. .. contents:: :depth: 3 :backlinks: none .. _configuration_types: Supported configuration types ----------------------------- Configuring EasyBuild can be done by: * using ``eb`` with **command line arguments** * setting **environment variables** (``$EASYBUILD_...``) * providing one or more **configuration files** Of course, combining any of these types of configuration works too (and is even fairly common). The order of preference for the different configuration types is as listed above, that is: * environment variables override the corresponding entries in the configuration file * command line arguments in turn override the corresponding environment variables *and* matching entries in the configuration file .. _configuration_consistency: Consistency across supported configuration types ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Note that the various available configuration options are handled **consistently** across the supported configuration types. For example: to configure EasyBuild to use Lmod as modules tool, the following alternatives are available: * configuration file entry (key-value assignment): .. code:: ini [config] modules-tool = Lmod * environment variable (upper case, ``EASYBUILD_`` prefix, ``-``'s becomes ``_``'s): .. code:: $ export EASYBUILD_MODULES_TOOL=Lmod * command line argument (long options preceded by ``--`` and (optionally) using ``=``): .. code:: $ eb --modules-tool=Lmod or .. code:: $ eb --modules-tool Lmod For more details w.r.t. each of the supported configuration types, see below. .. _configuration_file: Configuration file(s) ~~~~~~~~~~~~~~~~~~~~~ .. _list_of_configuration_files: List of used configuration files ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The list of configuration files that will be used by EasyBuild is determined in the following order of preference: * the path(s) specified via the **command line argument** ``--configfiles`` * the path(s) specified via the ``$EASYBUILD_CONFIGFILES`` **environment variable** * the **default paths** for EasyBuild configuration files .. _default_configuration_files: Default configuration files +++++++++++++++++++++++++++ By default, EasyBuild will use existing configuration files at the following paths: * ``$dir/easybuild.d/*.cfg``, for each directory ``$dir`` listed in ``$XDG_CONFIG_DIRS`` (where ``$XDG_CONFIG_DIRS`` defaults to ``/etc``) * ``$XDG_CONFIG_HOME/easybuild/config.cfg`` (where ``$XDG_CONFIG_HOME`` defaults to ``$HOME/.config``) Hence, if ``$XDG_CONFIG_HOME`` and ``$XDG_CONFIG_DIRS`` are not defined, EasyBuild will only consider default configuration files at ``/etc/easybuild.d/*.cfg`` and ``$HOME/.config/easybuild/config.cfg``. The configuration file located in ``$XDG_CONFIG_HOME`` will be listed *after* the ones obtained via ``$XDG_CONFIG_DIRS``, such that user-defined configuration settings have preference over system defaults. A detailed overview of the list of default configuration files is available via ``eb --show-default-configfiles`` (available since EasyBuild v2.1.0). For example:: $ XDG_CONFIG_DIRS=/tmp/etc:/tmp/moreetc eb --show-default-configfiles Default list of configuration files: [with $XDG_CONFIG_HOME: (not set), $XDG_CONFIG_DIRS: /tmp/etc:/tmp/moreetc] * user-level: ${XDG_CONFIG_HOME:-$HOME/.config}/easybuild/config.cfg -> /home/example/.config/easybuild/config.cfg => found * system-level: ${XDG_CONFIG_DIRS:-/etc}/easybuild.d/*.cfg -> {/tmp/etc, /tmp/moreetc}/easybuild.d/*.cfg => /tmp/etc/easybuild.d/config.cfg, /tmp/moreetc/easybuild.d/bar.cfg, /tmp/moreetc/easybuild.d/foo.cfg Default list of existing configuration files (4): /tmp/etc/easybuild.d/config.cfg, /tmp/moreetc/easybuild.d/bar.cfg, /tmp/moreetc/easybuild.d/foo.cfg, /home/example/.config/easybuild/config.cfg Multiple configuration files ++++++++++++++++++++++++++++ If multiple configuration files are listed via a mechanism listed above, the settings in the last configuration file have preference over the others. Each available configuration file will be used, and the configuration settings specified in these files will be retained according to the order of preference as indicated above: settings in configuration files specified via ``--configfiles`` override those in configuration files specified via ``$EASYBUILD_CONFIGFILES``, which in turns override settings in default configuration files. Ignored configuration files +++++++++++++++++++++++++++ On top of this, the ``--ignoreconfigfiles`` configuration option allows to specify configuration files that should be *ignored* by EasyBuild (regardless of whether they are specified via any of the options above). Configuration file format ^^^^^^^^^^^^^^^^^^^^^^^^^ The EasyBuild configuration file follows the default Python configuration format as parsed by the ``configparser`` module (see `http://docs.python.org/2/library/configparser.html`_). Configuration files are organized in sections, the section name for a particular configuration setting is indicated in the output of ``eb --help``. Some examples sections are: ``MAIN``, ``basic``, ``config``, ``informative``, ``override``, ``regtest``, ``software``, ``unittest``, etc. Sections are indicated by specifying the section name in square brackets on a dedicated line, e.g., ``[basic]``. Configuration settings are specified in a ``key = value`` or ``key: value`` format, **without using quotes for string-like values**. For boolean configuration settings, values that evaluated to ``True`` (e.g., ``true``, ``1``, …) are all equivalent to enabling the setting. Comment lines start with a hash character ``#`` (just like in Python code). An example configuration file that should make everything clear is shown below. .. code:: python [basic] # always enable logging to stdout logtostdout = true [config] # use Lmod as modules tool modules-tool: Lmod # use different default installation path prefix=/home/you/work/easybuild/ .. _configuration_file_templates_constants: Templates and constants supported in configuration files ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Two types of template values ``%(...)s`` are supported in configuration files: * for configuration options defined in the configuration file (and only those) * *syntax:* ``%(opt)s``, i.e., using the (lowercase) name of the configuration option * for the default value of selected configuration options (see ``eb --avail-cfgfile-constants``) * *syntax:* ``%(DEFAULT_OPT)s``, i.e., using the uppercase name of the configuration option and prefixed with ``DEFAULT_`` .. note:: These template values are only supported in configuration files, *not* in environment variable values or command line option values. .. note:: Using an unknown template value, i.e. either one for a configuration option that was not defined in the configuration file, or a non-existing one for a particular default value, will result in an error like: ``ConfigParser.InterpolationMissingOptionError: Bad value substitution``. Example +++++++ To include both the (custom) location for the easyconfigs archive repository and the default list of robot search paths in the active robot search path, the following configuration file entry can be used, featuring the template for the ``repositorypath`` configuration option and the provided ``DEFAULT_ROBOT_PATHS`` constant:: [basic] repositorypath = /home/example/easybuild/easyconfigs_archive robot-paths = %(repositorypath)s:%(DEFAULT_ROBOT_PATHS)s See also :ref:`controlling_robot_search_path`. Generating a template configuration file ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Since EasyBuild v1.10, a command line option ``--confighelp`` is available that prints out the help text as an annotated configuration file. This can be used as an empty template configuration file: .. code:: shell-session $ mkdir -p $HOME/.config/easybuild $ eb --confighelp > $HOME/.config/easybuild/config.cfg .. code:: shell-session $ head $HOME/.easybuild/config.cfg [MAIN] # Enable debug log mode (def False) #debug= # Enable info log mode (def False) #info= # Enable info quiet/warning mode (def False) #quiet= [basic] # Print build overview incl. dependencies (full paths) (def False) .. _configuration_env_vars: Environment variables ~~~~~~~~~~~~~~~~~~~~~ All configuration settings listed as long options in ``eb --help`` can also be specified via ``EASYBUILD_``-prefixed environment variables. Configuration settings specified this way always override the corresponding setting specified in a configuration file. For example, to enable debug logging using an environment variable: .. code:: shell-session $ export EASYBUILD_DEBUG=1 More examples of using environment variables to configure EasyBuild are shown in the sections below. .. tip:: Any configuration option of EasyBuild which can be tuned by command line or via the configuration file, can also be tuned via a corresponding environment variable. .. note:: If any ``$EASYBUILD``-prefixed environment variables are defined that do not correspond to a known configuration option, EasyBuild will report an error message and exit. .. _configuration_cmdline: Command line arguments ~~~~~~~~~~~~~~~~~~~~~~ The configuration type with the highest precedence are the ``eb`` command line arguments, which override settings specified through environment variables or in configuration files. For some configuration options, both short and long command line arguments are available (see ``eb --help``); the long options indicate how the configuration setting should be specified in a configuration file or via an environment variable (``$EASYBUILD_``). For boolean configuration settings, both the ``--