Skip to content

EasyBuild on Cray

As of EasyBuild v2.7.0, the support for using EasyBuild on Cray systems is considered stable. Tt enables building/installing software using the PrgEnv modules provided by Cray. This page provides an overview of the current status.

For more information about this, contact:

Thanks to:

  • Olli-Pekka Letho (CSC.fi)
  • Tim Robinson and Guilherme Peretti-Pezzi (CSCS.ch)
  • Eric Bavier (Cray)
  • Brett Bode (NCSA)

for providing us access to Cray systems, for their support and for testing and contributing to this work.

Test systems

EasyBuild toolchains

  • CrayCCE: PrgEnv-cray with pinned versions of cce, cray-libsci and cray-mpich
  • CrayGNU: PrgEnv-gnu with pinned versions of gcc, cray-libsci and cray-mpich
  • CrayIntel: PrgEnv-intel with pinned versions of intel, cray-libsci and cray-mpich
  • CrayPGI: PrgEnv-pgi with pinned versions of pgi and cray-mpich

versions:

  • Cray{CCE,GNU,Intel}/2015.06 (requires Cray PE June/2015)
  • Cray{CCE,GNU,Intel}/2015.11 (requires Cray PE November/2015)
  • CrayGNU/2016.03 (requires Cray PE March/2016)
  • Cray{GNU,PGI}/2016.04 (requires Cray PE April/2016)
  • Cray{GNU,Intel}/2016.06 (requires Cray PE June/2016)

What works already?

(see below for more information)

  • HPL (LINPACK) benchmark version 2.1

Major scientific software applications

  • CP2K 2.6.0
  • GROMACS 4.6.7
  • Python 2.7.9 + numpy 1.9.2 + scipy 0.15.1
  • WRF 3.6.1 (pending on Sisu)

An up-to-date list of software applications built on Cray systems at CSCS can be found https://github.com/eth-cscs/production/, see https://github.com/eth-cscs/production/tree/master/jenkins-builds.

Required EasyBuild configuration

Modules tool

  • Sisu: self-installed Lmod 5.8
  • Piz Daint, Dora, Swan, Santis, Brisi: system-provided environment modules 3.2.10

Example for environment modules 3.2.10:

source /opt/modules/3.2.10.3/init/bash
export PATH=/opt/modules/3.2.10.3/bin/:$PATH
export EASYBUILD_MODULES_TOOL=EnvironmentModulesC
export EASYBUILD_MODULE_SYNTAX=Tcl

Architecture

  • the craype-<target> module to load must be specified using --optarch
    • e.g., --optarch=sandybridge results in craype-sandybridge being loaded in the build environment used by EasyBuild

You can also export this option as a shell variable. Example for sandybridge:

export EASYBUILD_OPTARCH=sandybridge

Metadata for Cray-provided modules

  • Easybuild provides a sample metadata file in order to use modules provided by Cray:
easybuild-framework/etc/cray_external_modules_metadata.cfg

This file is loaded by default and contains enough information to build the easyconfig files shipped with EasyBuild.

If you need to use a customized file, it can be specified using --external-modules-metadata. For more details see Metadata for external modules.

Major supported/tested applications

(in alphabetical order)

CP2K

eb CP2K-2.6.0-CrayGNU-2015.06.eb -dr

GROMACS

eb GROMACS-4.6.7-CrayGNU-2015.06-mpi.eb -dr

HPL

eb HPL-2.1-CrayCCE-2015.06.eb -dr
eb HPL-2.1-CrayGNU-2015.06.eb -dr 
eb HPL-2.1-CrayIntel-2015.06.eb -dr

Python + numpy/scipy

eb Python-2.7.9-CrayGNU-2015.06.eb -dr 
# includes a few python packages (such as mpi4py, numpy and scipy)

WRF

eb WRF-3.6.1-CrayGNU-2015.06-dmpar.eb --dr