Skip to content

github

Utility module for working with github

Authors:

  • Jens Timmerman (Ghent University)
  • Kenneth Hoste (Ghent University)
  • Toon Willems (Ghent University)

GithubError

Bases: Exception

Error raised by the Githubfs

Githubfs

Bases: object

This class implements some higher level functionality on top of the Github api

__init__(githubuser, reponame, branchname=None, username=None, password=None, token=None)

Construct a new githubfs object

PARAMETER DESCRIPTION
githubuser

the github user's repo we want to use.

reponame

The name of the repository we want to use.

branchname

Then name of the branch to use (defaults to 'main' for easybuilders org, 'master' otherwise)

DEFAULT: None

username

(optional) your github username.

DEFAULT: None

password

(optional) your github password.

DEFAULT: None

token

(optional) a github api token.

DEFAULT: None

get_path(path)

returns the path as a Github object (from agithub)

get_repo()

Returns the repo as a Github object (from agithub)

isdir(githubobj) staticmethod

Check if this path points to a directory

isfile(githubobj) staticmethod

Check if this path points to a file

join(*args) staticmethod

This method joins 'paths' inside a github repository

listdir(path)

List the contents of a directory

read(path, api=True)

Read the contents of a file and return it Or, if api=False it will download the file and return the location of the downloaded file

walk(top=None, topdown=True)

Walk the github repo in an os.walk like fashion.

add_pr_labels(pr, branch=GITHUB_DEVELOP_BRANCH)

Try to determine and add labels to PR.

PARAMETER DESCRIPTION
pr

pull request number in easybuild-easyconfigs repo

branch

easybuild-easyconfigs branch to compare with

DEFAULT: GITHUB_DEVELOP_BRANCH

check_github()

Check status of GitHub integration, and report back. * check whether GitHub username is available * check whether a GitHub token is available, and whether it works * check whether git and GitPython are available * check whether push access to own GitHub repositories works * check whether creating gists works * check whether location to local working directories for Git repositories is available (not strictly needed)

check_online_status()

Check whether we currently are online Return True if online, else a list of error messages

check_pr_eligible_to_merge(pr_data)

Check whether PR is eligible for merging.

PARAMETER DESCRIPTION
pr_data

PR data obtained through GitHub API

RETURNS DESCRIPTION

boolean value indicates whether PR is eligible

close_pr(pr, motivation_msg=None)

Close specified pull request

PARAMETER DESCRIPTION
pr

PR number

motivation_msg

string containing motivation for closing the PR

DEFAULT: None

create_gist(txt, fn, descr=None, github_user=None, github_token=None)

Create a gist with the provided text.

create_remote(git_repo, account, repo, https=False)

Create remote in specified git working directory for specified account & repository.

PARAMETER DESCRIPTION
git_repo

git.Repo instance to use (after init_repo & setup_repo)

account

GitHub account name

repo

repository name

https

use https:// URL rather than git@

DEFAULT: False

delete_gist(gist_id, github_user=None, github_token=None)

Delete gist with specified ID.

det_account_branch_for_pr(pr_id, github_user=None, pr_target_repo=None)

Determine account & branch corresponding to pull request with specified id.

det_commit_status(account, repo, commit_sha, github_user)

Determine status of specified commit (pending, error, failure, success)

We combine two different things here:

  • the combined commit status (Travis CI sets a commit status)
  • results of check suites (set by CI run in GitHub Actions)

det_patch_specs(patch_paths, file_info, ec_dirs)

Determine software names for patch files

det_pr_labels(file_info, pr_target_repo)

Determine labels for a pull request based on provided information on files changed by that pull request.

det_pr_target_repo(paths)

Determine target repository for pull request from given cagetorized list of files

PARAMETER DESCRIPTION
paths

paths to categorized lists of files (easyconfigs, files to delete, patches, .py files)

download_repo(repo=GITHUB_EASYCONFIGS_REPO, branch=None, account=GITHUB_EB_MAIN, path=None, github_user=None)

Download entire GitHub repo as a tar.gz archive, and extract it into specified path.

PARAMETER DESCRIPTION
repo

repo to download

DEFAULT: GITHUB_EASYCONFIGS_REPO

branch

branch to download

DEFAULT: None

account

GitHub account to download repo from

DEFAULT: GITHUB_EB_MAIN

path

path to extract to

DEFAULT: None

github_user

name of GitHub user to use

DEFAULT: None

fetch_easyblocks_from_pr(pr, path=None, github_user=None)

Fetch patched easyconfig files for a particular PR.

fetch_easyconfigs_from_pr(pr, path=None, github_user=None)

Fetch patched easyconfig files for a particular PR.

fetch_files_from_pr(pr, path=None, github_user=None, github_account=None, github_repo=None)

Fetch patched files for a particular PR.

fetch_github_token(user)

Fetch GitHub token for specified user from keyring.

fetch_latest_commit_sha(repo, account, branch=None, github_user=None, token=None)

Fetch latest SHA1 for a specified repository and branch.

PARAMETER DESCRIPTION
repo

GitHub repository

account

GitHub account

branch

branch to fetch latest SHA1 for

DEFAULT: None

github_user

name of GitHub user to use

DEFAULT: None

token

GitHub token to use

DEFAULT: None

RETURNS DESCRIPTION

latest SHA1

fetch_pr_data(pr, pr_target_account, pr_target_repo, github_user, full=False, **parameters)

Fetch PR data from GitHub

find_easybuild_easyconfig(github_user=None)

Fetches the latest EasyBuild version eb file from GitHub

PARAMETER DESCRIPTION
github_user

name of GitHub user to use when querying GitHub

DEFAULT: None

find_software_name_for_patch(patch_name, ec_dirs)

Scan all easyconfigs in the robot path(s) to determine which software a patch file belongs to

PARAMETER DESCRIPTION
patch_name

name of the patch file

ec_dirs

list of directories to consider when looking for easyconfigs

RETURNS DESCRIPTION

name of the software that this patch file belongs to (if found)

github_api_get_request(request_f, github_user=None, token=None, **kwargs)

Helper method, for performing get requests to GitHub API.

PARAMETER DESCRIPTION
request_f

function that should be called to compose request, providing a RestClient instance

github_user

GitHub user name (to try and obtain matching GitHub token if none is provided)

DEFAULT: None

token

GitHub token to use

DEFAULT: None

RETURNS DESCRIPTION

tuple with return status and data

github_api_put_request(request_f, github_user=None, token=None, **kwargs)

Helper method, for performing put requests to GitHub API.

PARAMETER DESCRIPTION
request_f

function that should be called to compose request, providing a RestClient instance

github_user

GitHub user name (to try and obtain matching GitHub token if none is provided)

DEFAULT: None

token

GitHub token to use

DEFAULT: None

RETURNS DESCRIPTION

tuple with return status and data

init_repo(path, repo_name, silent=False)

Initialize a new Git repository at the specified location.

PARAMETER DESCRIPTION
path

location where Git repository should be initialized

repo_name

name of Git repository

silent

keep quiet (don't print any messages)

DEFAULT: False

install_github_token(github_user, silent=False)

Install specified GitHub token for specified user.

PARAMETER DESCRIPTION
github_user

GitHub user to install token for

silent

keep quiet (don't print any messages)

DEFAULT: False

is_patch_for(patch_name, ec)

Check whether specified patch matches any patch in the provided EasyConfig instance.

list_prs(params, per_page=GITHUB_MAX_PER_PAGE, github_user=None)

List pull requests according to specified selection/order parameters

PARAMETER DESCRIPTION
params

3-tuple with selection parameters for PRs (, , ), see https://developer.github.com/v3/pulls/#parameters

merge_pr(pr)

Merge specified pull request

new_branch_github(paths, ecs, commit_msg=None)

Create new branch on GitHub using specified files

PARAMETER DESCRIPTION
paths

paths to categorized lists of files (easyconfigs, files to delete, patches, files with .py extension)

ecs

list of parsed easyconfigs, incl. for dependencies (if robot is enabled)

commit_msg

commit message to use

DEFAULT: None

new_pr(paths, ecs, title=None, descr=None, commit_msg=None)

Open new pull request using specified files

PARAMETER DESCRIPTION
paths

paths to categorized lists of files (easyconfigs, files to delete, patches)

ecs

list of parsed easyconfigs, incl. for dependencies (if robot is enabled)

title

title to use for pull request

DEFAULT: None

descr

description to use for description

DEFAULT: None

commit_msg

commit message to use

DEFAULT: None

new_pr_from_branch(branch_name, title=None, descr=None, pr_target_repo=None, pr_metadata=None, commit_msg=None)

Create new pull request from specified branch on GitHub.

pick_default_branch(github_owner)

Determine default name to use.

post_comment_in_issue(issue, txt, account=GITHUB_EB_MAIN, repo=GITHUB_EASYCONFIGS_REPO, github_user=None)

Post a comment in the specified PR.

post_pr_labels(pr, labels)

Update PR labels

pr_files_cache(func)

Decorator to cache result of fetch_files_from_pr.

push_branch_to_github(git_repo, target_account, target_repo, branch)

Push specified branch to GitHub from specified git repository.

PARAMETER DESCRIPTION
git_repo

git.Repo instance to use (after init_repo & setup_repo)

target_account

GitHub account name

target_repo

repository name

branch

name of branch to push

reasons_for_closing(pr_data)

Look for valid reasons to close PR by comparing with existing easyconfigs.

setup_repo(git_repo, target_account, target_repo, branch_name, silent=False, git_only=False)

Set up repository by checking out specified branch for specfied GitHub account/repository.

PARAMETER DESCRIPTION
git_repo

git.Repo instance

target_account

name of GitHub account that owns GitHub repository

target_repo

name of GitHib repository

branch_name

name of branch to check out

silent

keep quiet (don't print any messages)

DEFAULT: False

git_only

only use git@github.com repo URL, skip trying https://github.com first

DEFAULT: False

setup_repo_from(git_repo, github_url, target_account, branch_name, silent=False)

Set up repository by checking out specified branch from repository at specified URL.

PARAMETER DESCRIPTION
git_repo

git.Repo instance

github_url

URL to GitHub repository

target_account

name of GitHub account that owns GitHub repository at specified URL

branch_name

name of branch to check out

silent

keep quiet (don't print any messages)

DEFAULT: False

sync_branch_with_develop(branch_name)

Sync branch with specified name with current develop branch.

sync_pr_with_develop(pr_id)

Sync pull request with specified ID with current develop branch.

sync_with_develop(git_repo, branch_name, github_account, github_repo)

Sync specified branch with develop branch.

update_branch(branch_name, paths, ecs, github_account=None, commit_msg=None)

Update specified branch in GitHub using specified files

PARAMETER DESCRIPTION
paths

paths to categorized lists of files (easyconfigs, files to delete, patches)

github_account

GitHub account where branch is located

DEFAULT: None

ecs

list of parsed easyconfigs, incl. for dependencies (if robot is enabled)

commit_msg

commit message to use

DEFAULT: None

update_pr(pr_id, paths, ecs, commit_msg=None)

Update specified pull request using specified files

PARAMETER DESCRIPTION
pr_id

ID of pull request to update

paths

paths to categorized lists of files (easyconfigs, files to delete, patches)

ecs

list of parsed easyconfigs, incl. for dependencies (if robot is enabled)

commit_msg

commit message to use

DEFAULT: None

validate_github_token(token, github_user)

Check GitHub token: * see if it conforms expectations (only [a-f]+[0-9] characters, length of 40) * see if it can be used for authenticated access