Integration with GitHub¶
EasyBuild provides several features that integrate with GitHub, where the different EasyBuild repositories are located.
From the EasyBuild command line eb
several options are available to
reach out to GitHub, which are documented below.
Requirements¶
Depending on which GitHub integration features you want to use, there are a couple of requirements:
- a GitHub account
- see https://github.com; creating an account is free
- a GitHub user name
- only required for authenticated access to the GitHub API, which can help to avoid rate limitations
- not (strictly) necessary for read-only operations, including:
- see Providing a GitHub username
- a GitHub token +
keyring
Python package- install via
pip install keyring
(for Python2:pip install 'keyring<19.0'
) - optionally install potentially unsafe keyrings:
pip install keyrings.alt
(but read and understand the warning) - allows accessing the GitHub API with authentication
- only strictly required for features that require GitHub 'write'
permissions
- i.e. for Uploading test reports and Submitting pull requests
- not used for Using easyconfig from commits
- see Installing a GitHub token
- install via
git
command /GitPython
Python package- install via
pip install GitPython
(for Python2:pip install 'GitPython<3.0'
) - only required when local
git
commands need to be executed, e.g. to manipulate a Git repository- i.e. for Submitting pull requests and Updating existing pull requests
- install via
- SSH public key registered on GitHub
- only required when
push
access to Git repositories that reside on GitHub is required- i.e. for Submitting pull requests and Updating existing pull requests
- see https://github.com/settings/ssh
- only required when
- fork of the EasyBuild repositories on GitHub
- only required for submitting/updating pull requests (Submitting pull requests and Updating existing pull requests
- see
Fork
button (top right) at https://github.com/easybuilders/easybuild-easyconfigs (for example)
See also Checking status of GitHub integration.
Configuration¶
The following sections discuss the EasyBuild configuration options relevant to the GitHub integration features.
Providing a GitHub username¶
(--github-user
)
To specify your GitHub username, do one of the following:
- use the
--github-user
configuration option on theeb
command line - define the
$EASYBUILD_GITHUB_USER
environment variable - specify
github-user
in your EasyBuild configuration file
(see also Configuring EasyBuild)
Installing a GitHub token¶
(--install-github-token
)
Note
requires: GitHub username + keyring
Python package
A GitHub token is a string of 40 characters that is tied to your GitHub account, allowing you to access the GitHub API authenticated.
Using a GitHub token is beneficial with respect to rate limitations, and enables write permissions on GitHub (e.g. posting comments, creating gists, opening pull requests).
To obtain a GitHub token:
- visit https://github.com/settings/tokens/new and log in with your GitHub account
- enter a token description, for example: "
EasyBuild
" - make sure (only) the
gist
andpublic_repo
(in therepo
section) scopes are fully enabled - click
Generate token
- copy-paste the generated token
Note
You will only be able to copy-paste the generated token right after you have created it.
The value corresponding to an existing token can not be retrieved
later through the GitHub interface.
Please keep your token secret at all times; it allows fully authenticated access to your GitHub account!
You can install the GitHub token in your keyring using EasyBuild, so it
can pick it up when it needs to, using eb --install-github-token
:
$ eb --github-user example --install-github-token
Token: <copy-paste-your-40-character-token-here>
Validating token...
Token seems to be valid, installing it.
Token 'e3a..0c2' installed!
EasyBuild will validate the provided token, to check that authenticated access to your GitHub account works as expected.
Note
EasyBuild will never print the full token value, to avoid leaking it.
For debugging purposes, only the first and last 3 characters will be
shown.
Specify location of working directories¶
(--git-working-dirs-path
)
You can specify the location of your Git working directories with one of the following:
- use the
--git-working-dirs-path
configuration option on theeb
command line - define the
$EASYBUILD_GIT_WORKING_DIRS_PATH
environment variable - specify the
git-working-dirs-path
option in your EasyBuild configuration file
The provided path should be the parent directory of the location of
the working directories (i.e. clones) of the EasyBuild repositories
(easybuild-easyconfigs
, etc.); the assumption is that you keep them
all in a single parent directory.
Although not strictly required, this is useful for speeding up
--new-pr
and --update-pr
, since it allows that the repository can be
copied & updated, rather than being cloned from scratch.
Checking status of GitHub integration¶
(--check-github
)
To check the status of your setup w.r.t. GitHub integration, the
--check-github
command line option can be used.
Using this will trigger EasyBuild to perform a number of checks, and report back on what the test results mean for the different GitHub integration features.
If all requirements are taken care of in your setup, you should see output like this:
$ eb --check-github
== temporary log file in case of crash /tmp/eb-xWCpWl/easybuild-hGnKS5.log
Checking status of GitHub integration...
Making sure we're online... OK
* GitHub user... example => OK
* GitHub token... e3f..0c8 (len: 40) => OK (validated)
* git command... OK ("git version 2.7.4 (Apple Git-66); ")
* GitPython module... OK
* push access to example/easybuild-easyconfigs repo @ GitHub... OK
* creating gists... OK
* location to Git working dirs... OK (/home/example/git-working-dirs)
All checks PASSed!
Status of GitHub integration:
* --from-pr: OK
* --new-pr: OK
* --review-pr: OK
* --update-pr: OK
* --upload-test-report: OK
Note
Checking whether push access to GitHub works may take some time, since a recent clone of
the easybuild-easyconfigs GitHub repository will be created in the
process (at a temporary location).
See also Requirements.
Using easyconfigs from pull requests¶
(--from-pr
, supported since EasyBuild v1.13.0)
Via the --from-pr
command line option (available since EasyBuild
v1.13.0), easyconfig files that are added or modified by a particular
pull request to the easybuild-easyconfigs GitHub
repository can
be used (regardless of whether the pull request is merged or not).
This can be useful to employ easyconfig files that are not available yet
in the active EasyBuild installation, or to test new contributions by
combining --from-pr
with --upload-test-report
(see Uploading test reports).
When --from-pr
is used, EasyBuild will download all modified files
(easyconfig files and patches) to a temporary directory before
processing them.
For example, to use the GCC v4.9.2 easyconfigs contributed via easyconfigs pull request #1177:
$ eb --from-pr 1177 --dry-run
== temporary log file in case of crash /tmp/eb-88quZc/easybuild-62fFdo.log
Dry run: printing build status of easyconfigs and dependencies
* [ ] /tmp/eb-88quZc/files_pr1177/GCC-4.9.2-CLooG-multilib.eb (module: GCC/4.9.2-CLooG-multilib)
* [ ] /tmp/eb-88quZc/files_pr1177/GCC-4.9.2-CLooG.eb (module: GCC/4.9.2-CLooG)
* [ ] /tmp/eb-88quZc/files_pr1177/GCC-4.9.2.eb (module: GCC/4.9.2)
== temporary log file /tmp/eb-88quZc/easybuild-62fFdo.log has been removed.
== temporary directory /tmp/eb-88quZc has been removed.
Note
To avoid GitHub rate limiting, let EasyBuild know which GitHub account should be used to query the GitHub API, and provide a matching GitHub token; see also Installing a GitHub token.
Relation between pull requests and current develop
branch¶
Since EasyBuild v2.9.0, the current develop
branch of the central
easybuild-easyconfigs
GitHub repository is taken into account when
applicable with --from-pr
. Before, only the branch corresponding to
the specified pull request itself was being considered, which
potentially did not reflect the correct state of things, in particular
for pull requests based on an outdated branch in which easyconfigs are
changed that have been updated in develop
as well.
As such, the exact semantics of --from-pr
depends on the state of the
specified pull request, i.e. whether or not the pull request was merged
already, whether the pull request is mergeable and stable (as indicated
by GitHub Actions), etc.
Open stable pull requests¶
For open pull requests that are stable (i.e. tests pass and no merge
conflicts), the pull request is effectively treated as a patch to the
current develop
branch. This is done to ensure that contributions that
are picked up via --from-pr
are correctly evaluated.
First, the current develop
branch of the central
easybuild-easyconfigs
GitHub repository is downloaded to a temporary
directory. Afterwards, the patch corresponding to the specified pull
request is applied on top of the develop
branch. This results in a
correct reflection of how the easyconfig files would look like if the
pull request would be merged, which is particularly important for
testing of contributions (see also Uploading test reports).
Easyconfig files touched by the pull request that are explicitly specified are then picked up from this location; see also Specifying particular easyconfig files.
Merged pull requests¶
For merged pull requests, the current develop
branch is considered to
be the correct state of the easyconfigs touched by the pull request.
Note that this implies that the easyconfig files being picked up are
potentially different from the ones that appear in the specified pull
request itself, taking into account that further updates may have been
applied in the develop
branch since the pull request got merged.
Closed or unstable pull requests¶
For closed and unstable pull requests, only the branch corresponding to
the pull request itself is being considered, which aligns with the
semantics of --from-pr
as it was before EasyBuild v2.9.0. In this
case, the current develop
branch is not taken into account.
Note
A pull request is considered unstable when GitHub reports merge conflicts or when GitHub Actions reports
one or more failing tests.
Synergy with --robot
¶
Since EasyBuild v1.15.0, the temporary directory containing the easyconfigs (and patch files) from the specified pull request is included in the robot search path.
Up until EasyBuild v2.9.0, this directory was prepended to the robot
search path, to ensure that easyconfigs that were modified in the
respective pull request are picked up via --robot
when they are
required. Thus, for easyconfig files that were available in the pull
request as well as locally, the ones from the specified pull request
were preferred.
This was changed in EasyBuild v2.9.0, where the directory containing the
easyconfigs touched by the pull request is appended to the robot
search path. This change was made to ensure that customized easyconfig
files that are available in the robot search path are preferred over the
(patched) easyconfig files from the develop
branch (see also
Relation between pull requests and current develop
branch).
For example, to build and install HPL
with the intel/2015a
toolchain, both of which are contributed via easyconfigs pull request
#1238:
$ eb --from-pr 1238 --dry-run --robot $HOME/easyconfigs
== temporary log file in case of crash /tmp/eb-A1fRvw/easybuild-Eqc8Oi.log
Dry run: printing build status of easyconfigs and dependencies
* [x] /home/example/easyconfigs/g/GCC/GCC-4.9.2.eb (module: GCC/4.9.2)
* [x] /home/example/easyconfigs/i/icc/icc-2015.1.133-GCC-4.9.2.eb (module: icc/2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/ifort/ifort-2015.1.133-GCC-4.9.2.eb (module: ifort/2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/iccifort/iccifort-2015.1.133-GCC-4.9.2.eb (module: iccifort/2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/impi/impi-5.0.2.044-iccifort-2015.1.133-GCC-4.9.2.eb (module: impi/5.0.2.044-iccifort-2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/iimpi/iimpi-7.2.3-GCC-4.9.2.eb (module: iimpi/7.2.3-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/imkl/imkl-11.2.1.133-iimpi-7.2.3-GCC-4.9.2.eb (module: imkl/11.2.1.133-iimpi-7.2.3-GCC-4.9.2)
* [ ] /tmp/eb-A1fRvw/files_pr1238/intel-2015a.eb (module: intel/2015a)
* [ ] /tmp/eb-A1fRvw/files_pr1238/HPL-2.1-intel-2015a.eb (module: HPL/2.1-intel-2015a)
== temporary log file /tmp/eb-A1fRvw/easybuild-Eqc8Oi.log has been removed.
== temporary directory /tmp/eb-A1fRvw has been removed.
Note that the easyconfigs that are required to resolve dependencies and
are available locally in $HOME/easyconfigs
are being picked up as
needed.
Specifying particular easyconfig files¶
Since EasyBuid v2.0.0 the particular easyconfigs to be used can be
specified, rather than using all easyconfigs that are touched by the
pull request (which is the default if no easyconfigs are specified
alongside --from-pr
).
For example, to only use CMake-3.0.0-intel-2015a.eb
from easyconfigs
pull request
#1330,
and ignore the other easyconfigs being contributed in that same pull
request for netCDF, WRF, ...:
$ eb --from-pr 1330 CMake-3.0.0-intel-2015a.eb --dry-run --robot $HOME/easyconfigs
== temporary log file in case of crash /tmp/eb-QhM_qc/easybuild-TPvMkJ.log
Dry run: printing build status of easyconfigs and dependencies
* [x] /home/example/easyconfigs/g/GCC/GCC-4.9.2.eb (module: GCC/4.9.2)
* [x] /home/example/easyconfigs/i/icc/icc-2015.1.133-GCC-4.9.2.eb (module: icc/2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/ifort/ifort-2015.1.133-GCC-4.9.2.eb (module: ifort/2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/iccifort/iccifort-2015.1.133-GCC-4.9.2.eb (module: iccifort/2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/impi/impi-5.0.2.044-iccifort-2015.1.133-GCC-4.9.2.eb (module: impi/5.0.2.044-iccifort-2015.1.133-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/iimpi/iimpi-7.2.3-GCC-4.9.2.eb (module: iimpi/7.2.3-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/imkl/imkl-11.2.1.133-iimpi-7.2.3-GCC-4.9.2.eb (module: imkl/11.2.1.133-iimpi-7.2.3-GCC-4.9.2)
* [x] /home/example/easyconfigs/i/intel/intel-2015a.eb (module: intel/2015a)
* [x] /home/example/easyconfigs/n/ncurses/ncurses-5.9-intel-2015a.eb (module: ncurses/5.9-intel-2015a)
* [ ] /tmp/eb-QhM_qc/files_pr1330/CMake-3.0.0-intel-2015a.eb (module: CMake/3.0.0-intel-2015a)
== temporary log file /tmp/eb-QhM_qc/easybuild-TPvMkJ.log has been removed.
== temporary directory /tmp/eb-QhM_qc has been removed.
Again, note that locally available easyconfigs that are required to resolve dependencies are being picked up as needed.
Using easyconfigs from a commit¶
(--from-commit
, supported since EasyBuild v4.9.1)
Using --from-commit
, you can let EasyBuild use easyconfig files that were added or modified in a specific Git commit.
This is similar to using --from-pr
(see Using easyconfigs from pull requests), but it differs in two ways:
-
The GitHub API is not used to obtain the easyconfig files from a particular commit. This can be important since it avoids hitting the GitHub API rate limits, especially when no GitHub token is available;
-
The easyconfig files will be exactly as they are in the specified commit and they can also be consulted on GitHub (link points to a fictitious commit SHA
c0ff3315c0ffeec0ff3315c0ffeec0ffeec0ff33
). This is not the case with--from-pr
, since then either the PR branch is merged with the target branch (typicallydevelop
) before obtained the easyconfigs (for open PRs), or thedevelop
branch is used (for merged PRs). This aspect can be important to ensure reproducibility.
Note
The commit SHA passed to --from-commit
must be a full commit SHA consisting of 40 characters,
not a the first part of a commit SHA as you often see in GitHub.
Example usage:
# use easyconfig file for EasyBuild v4.9.3 which was contributed in https://github.com/easybuilders/easybuild-easyconfigs/pull/21412
eb --from-commit 82206bb1c68d8d9ccd551b36efb0a0cc2251147c EasyBuild-4.9.3.eb
Using easyblocks from pull requests¶
(--include-easyblocks-from-pr
, supported since EasyBuild v4.2.0)
Via the --include-easyblocks-from-pr
command line option, easyblocks
that are added or modified by a particular pull request to the
easybuild-easyblocks GitHub repository can be
used (regardless of whether the pull request is merged or not).
This can be useful to employ easyblocks that are not available yet in
the active EasyBuild installation, or to test new contributions by
combining --include-easyblocks-from-pr
with --from-pr
and
--upload-test-report
(see Uploading test reports).
When --include-easyblocks-from-pr
is used, EasyBuild will download all
modified easyblocks to a temporary directory before processing them.
Just like with --include-easyblocks
(see Including additional easyblocks), the easyblocks that are
included are preferred over the ones included in the EasyBuild installation.
For example, to use the LAMMPS easyblock contributed via easyblocks pull request #1964 together with the LAMMPS v7Aug2019 easyconfigs contributed via easyconfigs pull request #9884:
$ eb --from-pr 9884 --include-easyblocks-from-pr 1964 --list-easyblocks=detailed
== temporary log file in case of crash /tmp/eb-Eq2zsJ/easybuild-1AaWf8.log
EasyBlock (easybuild.framework.easyblock)
...
| | |-- EB_LAMMPS (easybuild.easyblocks.lammps @ /tmp/included-easyblocks-rD2HEQ/easybuild/easyblocks/lammps.py)
...
Using easyblocks from a commit¶
(--include-easyblocks-from-commit
, supported since EasyBuild v4.9.1)
Using --include-easyblocks-from-commit
, you can let EasyBuild use easyblocks exactly
as they were in a particular commit.
This differs from using --include-easyblocks-from-pr
in the same way as using --from-commit
differs from using
--from-commit
, see also Using easyconfigs from a commit.
Uploading test reports¶
(--upload-test-report
, supported since EasyBuild v1.13.0)
Note
requires that a GitHub token was required gist
permissions is
available, cfr. Installing a GitHub token
For every installation performed with EasyBuild, a test report is generated. By default, the test report is copied in the installation directory, right next to the log file (see also Understanding EasyBuild logs).
Using --upload-test-report
, the test report can also be pushed to
GitHub (as a gist, cfr. https://gist.github.com) to share it with
others.
Each test report includes:
- an overview of the easyconfigs being processed
- time & date
- the exact
eb
command line that was used - the full EasyBuild configuration that was in place
- information about the system on which EasyBuild was used (hostname, OS, architecture, etc.)
- the list of modules that was loaded
- the full environment of the session in which
eb
was run (note: can be filtered, see Filtering the environment details
For each easyconfig that failed to install a partial log will be uploaded as a separate gist, and a link to this gist will be included in the test report.
If --upload-test-report
is combined with --from-pr
, a comment
referring to the test report (incl. a brief summary) will be placed in
the respective pull request. This makes it a very powerful tool when
testing contributions.
Note
If you want to easily access a test report without uploading it
to GitHub, use --dump-test-report
.
Example:
$ eb --from-pr 3153 --rebuild --upload-test-report
== temporary log file in case of crash /tmp/eb-aqk20q/easybuild-wuyZBV.log
== processing EasyBuild easyconfig /tmp/eb-aqk20q/files_pr3153/EasyBuild/EasyBuild-2.8.1.eb
== building and installing EasyBuild/2.8.1...
...
== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file /home/example/software/EasyBuild/2.8.1/easybuild/easybuild-EasyBuild-2.8.1-20160603.090702.log
== Test report uploaded to https://gist.github.com/1cb2db8a2913a1b8ddbf1c6fee3ff83c and mentioned in a comment in easyconfigs PR#3153
== Build succeeded for 1 out of 1
== Temporary log file(s) /tmp/eb-aqk20q/easybuild-wuyZBV.log* have been removed.
== Temporary directory /tmp/eb-aqk20q has been removed.
The resulting test report can be viewed at https://gist.github.com/1cb2db8a2913a1b8ddbf1c6fee3ff83c .
Note
It is common to use --rebuild
in combination with --upload-test-report
, to ensure that all easyconfigs
in the pull request are rebuilt, resulting in a complete test report.
Filtering the environment details¶
(--test-report-env-filter
)
Since the environment of the session in which eb
was used may contain
sensitive information, it can be filtered through
--test-report-env-filter
.
This configuration option takes a regular expression that is used to determine which environment variables can be included in the test report (based on their name). Environment variables for which the name matches the specified regular expression will not be included in the test report.
An example of a typical setting:
export EASYBUILD_TEST_REPORT_ENV_FILTER='^SSH|USER|HOSTNAME|UID|.*COOKIE.*'
Reviewing easyconfig pull requests¶
(--review-pr
)
A useful tool when reviewing pull requests for the
easybuild-easyconfigs
repository that
add new or update existing easyconfig files is --review-pr
.
The 'files' tab in the GitHub interface shows the changes being made to
existing files; using --review-pr
the differences with one or more
other similar easyconfig files, for example the one(s) with the same
toolchain (version) and/or software version, can also be evaluated.
This is very useful to quickly see how easyconfig files in pull requests differ from existing easyconfig files, and to maintain consistency across easyconfig files where desired.
The --review-pr
output consists of a 'multidiff' view per easyconfig
file that is being touched by the specified pull request. The exact
format of the output depends on whether EasyBuild is configured to allow
colored output (enabled by default, see --color
).
Search criteria for similar easyconfigs¶
The set of existing similar easyconfig files is determined by specific search criteria; the first one that results in a non-empty set of easyconfigs is retained.
The search criteria consists of a combination of the software version criterion with additional restrictions.
The software version criterion is one of the criterions below
(considered in order), with x.y.z
the software version of the
easyconfig file from the pull request:
- exact same software version
- same major/minor software version (same
x
andy
) - same major software version (same
x
) - no (partial) version match (so consider any version)
The addition restrictions are the following (also considered in order):
- matching versionsuffix and toolchain name/version
- matching versionsuffix and toolchain name (any toolchain version)
- matching versionsuffix (any toolchain name/version)
- matching toolchain name/version (any versionsuffix)
- matching toolchain name (any versionsuffix, toolchain version)
- no extra requirements (any versionsuffix, toolchain name/version)
Merging easyconfig pull requests¶
(--merge-pr
, supported since EasyBuild v3.3.1)
EasyBuild maintainers need to take the Requirements for pull requests into account.
They can merge a pull request to the easybuild-easyconfigs
repository
via eb --merge-pr
, which will first verify whether the pull request
meets the prescribed requirements (at least the ones that can be
verified automatically).
For example, for a pull request that is not eligible for merging yet:
$ eb --merge-pr 4725
== temporary log file in case of crash /tmp/eb-ba7rVp/easybuild-fBfcwN.log
easybuilders/easybuild-easyconfigs PR #4725 was submitted by vanzod, you are using GitHub account 'example'
Checking eligibility of easybuilders/easybuild-easyconfigs PR #4725 for merging...
* targets develop branch: OK
* test suite passes: FAILED => not eligible for merging!
* last test report is successful: (no test reports found) => not eligible for merging!
* approved review: MISSING => not eligible for merging!
* milestone is set: no milestone found => not eligible for merging!
WARNING: Review indicates this PR should not be merged (use -f/--force to do so anyway)
When a PR is considered eligible for merging, EasyBuild will go ahead and merge it:
$ eb --merge-pr 4829
== temporary log file in case of crash /tmp/eb-F9a3oB/easybuild-3B2wdq.log
easybuilders/easybuild-easyconfigs PR #4829 was submitted by SethosII, you are using GitHub account 'example'
Checking eligibility of easybuilders/easybuild-easyconfigs PR #4829 for merging...
* targets develop branch: OK
* test suite passes: OK
* last test report is successful: OK
* approved review: OK (by boegel)
* milestone is set: OK (3.3.1)
Review OK, merging pull request!
Adding comment to easybuild-easyconfigs issue #4829: 'Going in, thanks @SethosII!'
Merged easybuilders/easybuild-easyconfigs pull request #4829
Note
eb --merge-pr
can also be run in dry run mode, by also using one of the following options:
--dry-run
, -D
, --extended-dry-run
, -x
.
This results in the same checks being performed but skips the actual merging of the pull request, resulting in messages like:
$ eb --merge-pr 4829 --dry-run
...
Review OK, merging pull request!
[DRY RUN] Adding comment to easybuild-easyconfigs issue #4829: 'Going in, thanks @SethosII!'
[DRY RUN] Merged easybuilders/easybuild-easyconfigs pull request #4829
Submitting new and updating pull requests¶
(--new-pr
, --update-pr
, supported since EasyBuild v2.6.0)
EasyBuild provides two simple yet powerful features that make
contributing to the central EasyBuild repositories significantly easier
and less error-prone, especially for people who are not very familiar
with git
and/or GitHub yet:
--new-pr
to create new pull requests--update-pr
to update existing pull requests
Previewing easyconfig pull requests¶
(--preview-pr
, supported since EasyBuild v3.5.0)
It is very useful to quickly see how easyconfig files in pull requests differ from existing easyconfig files, and to maintain consistency across easyconfig files where desired.
Maintainers will use --review-pr
as part of the review process once
the PR is submitted (see Reviewing easyconfig pull requests), but it is now possible to
preview that output before submitting a PR, eventually fixing any
inconsistencies in advance.
To preview a PR before submitting, simply use --preview-pr
with the
list of files to submit:
eb --preview-pr example.eb example.patch
Besides accepting local files instead of a PR number, --preview-pr
works the same as --review-pr
, as described in Comparing with existing easyconfigs.
Submitting pull requests¶
(--new-pr
)
Note
Submitting pull requests using --new-pr
only works for the easybuild-easyconfigs
repository, for now.
For other repositories, see the manual procedure documented at Pull requests.
To create a new pull request, the --new-pr
command line option can be
used, provided that the necessary requirements are fulfilled (see
Requirements).
In its simplest form, you just provide the location of the file(s) that you want to include in the pull request:
eb --new-pr test.eb
This takes care of all the steps required to make a contribution, i.e.:
- set up a working copy of the relevant EasyBuild repository (e.g.,
easybuild-easyconfigs
) - create a new 'feature' branch, starting from the up-to-date
develop
branch - renaming easyconfig files according to their
name
,version
,versionsuffix
andtoolchain
- moving easyconfig files to the right location in the repository (e.g.
easybuild/easyconfigs/e/EasyBuild/
) - staging and committing the files in the feature branch
- pushing the feature branch to your fork of the relevant EasyBuild repository on GitHub
- creating the pull request, targeting the
develop
branch of the central EasyBuild repository (e.g.easybuilders/easybuild-easyconfigs
)
It should be clear that automating this whole procedure with a single
simple eb
command greatly lowers the bar for contributing, especially
since it even alleviates the need for interacting directly with git
entirely!
The working copy of the EasyBuild repository is created in a temporary location, and cleaned up once the pull request has been created. EasyBuild does not make changes to an existing working copy you may have in place already (cfr. Specify location of working directories).
Note
When modifying existing files via --new-pr
,
you must specify a (meaningful) commit message using
--pr-commit-msg
, see Controlling pull request metadata.
Example¶
For example, to create a pull request for a new version of, let's say, EasyBuild:
$ eb --new-pr example.eb
== temporary log file in case of crash /tmp/eb-mWKR9u/easybuild-cTpf2W.log
== copying /home/example/git-working-dirs/easybuild-easyconfigs...
== fetching branch 'develop' from https://github.com/easybuilders/easybuild-easyconfigs.git...
Opening pull request
* target: easybuilders/easybuild-easyconfigs:develop
* from: boegel/easybuild-easyconfigs:20160530131447_new_pr_EasyBuild281
* title: "{tools}[dummy/dummy] EasyBuild v2.8.1"
* description:
"""
(created using `eb --new-pr`)
"""
* overview of changes:
.../easyconfigs/e/EasyBuild/EasyBuild-2.8.1.eb | 35 ++++++++++++++++++++++
1 file changed, 35 insertions(+)
Opened pull request: https://github.com/easybuilders/easybuild-easyconfigs/pull/3153
Yes, it's that easy!
Updating existing pull requests¶
(--update-pr
)
Note
Updating pull requests using --update-pr
only works for the easybuild-easyconfigs
repository, for now.
For other repositories, see the manual procedure documented at Pull requests.
Similarly to creating new pull requests, existing pull requests can be
easily updated using eb --update-pr
(regardless of whether or not they
were created with --new-pr
).
The usage is equally simple, for example to update pull request #1234
just list the changed/new file(s):
eb --update-pr 1234 example.eb
Again, this take care of the whole procedure required to update an existing pull request:
- set up a working copy of the relevant EasyBuild repository (e.g.,
easybuild-easyconfigs
) - determining the branch corresponding to the pull request, which should be updated by pushing a new commit to it
- checking out that branch
- renaming easyconfig files according to their
name
,version
,versionsuffix
andtoolchain
- moving easyconfig files to the right location in the repository (e.g.
easybuild/easyconfigs/e/EasyBuild/
) - staging and committing the (changed/new) files
- pushing the updated branch to GitHub
Again, not a single git
command to be executed; the only thing that is
required is the ID of the pull request that should be updated.
Just like with --new-pr
, this is done in a temporary working copy of
the repository, no changes are made to a possible existing working copy.
Note
When using --update-pr
you must specify a (meaningful) commit message
via --pr-commit-msg
, see Controlling pull request metadata.
Example¶
For example, to update pull request #3153 with a changed easyconfig file:
$ eb --update-pr 3153 example.eb
== temporary log file in case of crash /tmp/eb-gO2wJu/easybuild-37Oo2z.log
== Determined branch name corresponding to easybuilders/easybuild-easyconfigs PR #3153: 20160530131447_new_pr_EasyBuild281
== copying /home/example/git-working-dirs/easybuild-easyconfigs...
== fetching branch '20160530131447_new_pr_EasyBuild281' from https://github.com/boegel/easybuild-easyconfigs.git...
Overview of changes:
easybuild/easyconfigs/e/EasyBuild/EasyBuild-2.8.1.eb | 3 +++
1 file changed, 3 insertions(+)
Updated easybuilders/easybuild-easyconfigs PR #3159 by pushing to branch boegel/20160530131447_new_pr_EasyBuild281
Including patch files in easyconfigs pull requests¶
Next to providing one or more easyconfig files to add/update via
--new-pr
or --update-pr
, you can also include patch files that are
required by those easyconfig files.
EasyBuild will try and figure out where each patch file should be located (i.e. in the same directory as the easyconfig files that require that patch file), by scanning the provided easyconfigs (or, if needed, scanning all existing easyconfig files).
For example:
eb --new-pr example.eb example.patch --pr-commit-msg "just an example"
Note
When providing one or more patch files, you must specify a (meaningful) commit message
via --pr-commit-msg
, see Controlling pull request
metadata.
Deleting easyconfig files or patches¶
Next to adding easyconfigs files or patches, or modifying existing ones,
you can also specify to delete particular files, by including a colon
character :
before the name of the file.
For example:
eb --new-pr :example-1.0.eb --pr-commit-msg "delete example-1.0.eb easyconfig file"
Note
When deleting existing files, you must specify a custom commit message using --pr-commit-msg
,
see also Controlling pull request metadata.
Controlling pull request metadata¶
You can control the metadata for pull requests using the following configuration options:
--pr-branch-name
: branch name for new pull requests--pr-commit-msg
: commit message to use when creating new or updating existing pull requests--pr-descr
: pull request description--pr-title
: pull request title
EasyBuild will use sensible defaults for each of these, see below.
Default branch name for new pull requests¶
The branch name for new pull requests will be composed from:
- a timestamp, down to the second in an attempt to make it unique
- example:
20160513141133
for a pull request created on May 13th 2016, 2:11:33 PM
- example:
- a label
new_pr
- the software name and version of the first easyconfig file, with some
filtering (e.g. remove
.
's)- example:
GCC530
for GCC v5.3.0
- example:
Full example: 20160513141133_new_pr_GCC530
Although there is usually no reason to change this default, it can be
done if desired using --pr-branch-name
when opening a new pull request
with --new-pr
.
Default commit message¶
EasyBuild will try to generate an appropriate default commit message
when only new easyconfigs are being added via --new-pr
.
When existing easyconfigs are being modified, patch files are being
added/updated or --update-pr
is used, a custom (meaningful) commit
message must be provided via --pr-commit-msg
(see Controlling pull request metadata).
Default pull request description¶
By default, the pull description will only contain the following text:
(created using eb --new-pr)
It is generally advised to provide more descriptive information, although the changes made by the pull request may be self-explanatory (e.g. when only adding new easyconfig files).
To change this default text, you can either use --pr-descr
or edit the
description via the GitHub interface after the pull request has been
opened.
Particularly useful information to specify here is dependencies on other
pull requests, by copy-pasting the respective URLs with a short
descriptive message like 'depends on PR <URL>
'.
Default pull request title¶
The pull request title is derived from the easyconfig files being
changed/added, taking into account the recommendation for easyconfig
pull requests to clearly specify module class, toolchain, software
name/version, as follows:
{<module_class>}[<toolchain>] <software_name> v<software_version>
.
For example, when opening a pull request for an easyconfig for Python
2.7.11 with the intel/2016a
toolchain, the default pull request title
will be something like: {lang}[intel/2016a] Python v2.7.11
.
If multiple easyconfig files are provided, the respective software
names/versions will be included separated by a ,
, up until the first 3
easyconfig files (to avoid excessively lengthy pull request titles).
In case (only) existing easyconfig files are being changed, it's
advisable to provide a more descriptive title using --pr-title
.
Configuring --new-pr
and --update-pr
¶
By default, --new-pr
and --update-pr
affect pull requests to the
central easybuilders/easybuild-easyconfigs
repository.
However, this can be changed with the following configurations options:
--pr-target-account
(default:easybuilders
): target GitHub account for new pull requests--pr-target-branch
(default:develop
): target branch for new pull requests--pr-target-repo
(default:easybuild-easyconfigs
): target repository for new pull requests
Synergy with --dry-run
/-D
and --extended-dry-run
/-x
¶
Both --new-pr
and --update-pr
are 'dry run-aware', in the sense that
you can combine them with either --dry-run
/-D-
or
--extended-dry-run
/-x
to preview the pull request they would
create/update without actually doing so.
For example:
$ eb --new-pr EasyBuild-2.9.0.eb -D
== temporary log file in case of crash /tmp/eb-1ny69k/easybuild-UR1Wr4.log
== copying /home/example/git-working-dirs/easybuild-easyconfigs...
== fetching branch 'develop' from https://github.com/easybuilders/easybuild-easyconfigs.git...
Opening pull request [DRY RUN]
* target: easybuilders/easybuild-easyconfigs:develop
* from: boegel/easybuild-easyconfigs:20160603105641_new_pr_EasyBuild290
* title: "{tools}[dummy/dummy] EasyBuild v2.9.0"
* description:
"""
(created using `eb --new-pr`)
"""
* overview of changes:
.../easyconfigs/e/EasyBuild/EasyBuild-2.9.0.eb | 35 ++++++++++++++++++++++
1 file changed, 35 insertions(+)
The only difference between using --dry-run
and --extended-dry-run
is that the latter will show the full diff of the changes (equivalent to
git diff
), while the former will only show a summary of the changes
(equivalent to git diff --stat
, see example above).