Page Comparison

Spack Introduction and Environment Setup

What is Spack?

Spack is a package manager, similar to yum, apt-get, or conda. It allows for multiple package versions/configurations.  Unlike the other package managers listed, Spack is specifically designed to be used on HPC resources with varying architectures.  The purpose of this documentation is to provide a practical guide to quickly enable you to take advantage of its use in your research.

How to set up your environment to use Spack

First, log in to one of the gateway nodes; here, we are logging onto pascal:

ssh username@pascal

If off campus, use this command:

ssh username@pascal.med.cornell.edu

Next, add the following code to your ~/.bashrc, which will automatically load Spack into your shell on all servers besides gateways:

# detect kernel then source spack file if it exists
KERNEL=$(uname -r | cut -d '.' -f1,2)
 
if [[ "${HOSTNAME}" != "pascal.med.cornell.edu" ]] || [[ "${HOSTNAME}" != "aphrodite.med.cornell.edu" ]] || [[ "${HOSTNAME}" != "aristotle.med.cornell.edu" ]]; then
 
    if [[ "${KERNEL}" == *"2.6.3"* ]] ; then
            if [ -f /softlib/apps/EL6/spack/share/spack/setup-env.sh ] ; then
                    . /softlib/apps/EL6/spack/share/spack/setup-env.sh
            fi
    fi
 
    if [[ "${KERNEL}" ==  *"3.10"* ]] ; then
            if [ -f /softlib/apps/EL7/spack/share/spack/setup-env.sh ] ; then
                    . /softlib/apps/EL7/spack/share/spack/setup-env.sh
            fi
    fi
 
fi

Package Information

How to show what packages are installed in Spack

The Spack Basic Usage guide demonstrates how to show which packages are already present in Spack though we will show them here as well.

To see all packages currently installed, use this command:

spack find

...

...

Spack Introduction and Environment Setup

What is Spack?

Spack is a package manager, similar to yum, apt-get, or conda. It allows for multiple package versions/configurations. Unlike the other package managers listed, Spack is specifically designed to be used on HPC resources with varying architectures.

The purpose of this documentation is to provide a practical guide to quickly enable you to take advantage of its use in your research.

How to set up your environment to use Spack

First, log in to one of the gateway nodes; here, we are logging onto pascal:

ssh username@pascal

If off campus, use this command:

ssh username@pascal.med.cornell.edu

Next, add the following code to your ~/.bashrc, which will automatically load Spack into your shell on all servers besides gateways:

# detect kernel version, then source spack file if it exists
_KERNEL_VERSION=$(uname -r | cut -d '.' -f1,2,3)
if [[ "${HOSTNAME}" != "pascal.med.cornell.edu" ]] || [[ "${HOSTNAME}" != "aphrodite.med.cornell.edu" ]] || [[ "${HOSTNAME}" != "aristotle.med.cornell.edu" ]]; then
    if [[ "${_KERNEL_VERSION}" == *"2.6.3"* ]] ; then
        if [ -f /softlib/apps/EL6/spack/share/spack/setup-env.sh ] ; then
            python@2.7.13
autoconf@2.69. /softlib/apps/EL6/spack/share/spack/setup-env.sh
        fftw@3.3.7fi
    fi
     libpciaccess@0.13.5   openmpi@3.0.1if [[ "${_KERNEL_VERSION}" ==  *"3.10"* ]] ; then
           python@2.7.14
autoconf@2.69    fftw@3.3.8if [ -f /software/spack/centos7/share/spack/setup-env.sh ] ; then
             libpng@1.6.27. /software/spack/centos7/share/spack/setup-env.sh
         openmpi@3.0.1   elif [ -f /software/spack/share/spack/setup-env.sh ] ; then
       python@3.4.3 automake@1.15    fftw@3.3.8         libpng@1.6.29  /software/spack/share/spack/setup-env.sh
        fi
 openmpi@3.0.1   fi
fi
builtin      python@3.5.2
automake@1.16.1  fftw@3.3.8         libpthread-stubs@0.4  openmpi@3.0.1           python@3.5.2
automake@1.16.1  flex@2.6.3         libsigsegv@2.11     unset _KERNEL_VERSION

...

Package Information

How to show what packages are installed in Spack

The Spack Basic Usage guide demonstrates how to show which packages are already present in Spack though we will show them here as well.

To see all packages currently installed, use this command:

spack find

==> 1027 installed packages.
-- linux-centos7-x86_64 / gcc@4.8.5 -----------------------------
autoconf@2.69    fftw@3.3.7         libjpeg-turbo@1.5.90  openmpi@3.0.1           python@3py-six@1.610.0
binutils@2autoconf@2.2869    fltk@1fftw@3.3.37         libtiff@4libpciaccess@0.013.94         openmpi@3.0.1.0           readline@7python@2.07.13
binutils@2autoconf@2.2869    fltk@1fftw@3.3.37         libpciaccess@0.13.5   libtool@2openmpi@3.40.61         openssl@1  python@2.7.14
autoconf@2.69    fftw@3.3.8        readline@7.0 bison@3libpng@1.06.427      font-util@1.3.1    libx11@1openmpi@3.60.51          openssl@1 python@3.4.3
automake@1.15    fftw@3.3.8         readline@7.0
boost@1.64.0libpng@1.6.29         fontconfig@2openmpi@3.110.1  libxau@1.0.8         python@3.5.2
openssl@1automake@1.016.2k1  fftw@3.3.8        relion@2.0.3
bowtie2@2.3.1    fontconfig@2.12.1  libxcb@1.13 libpthread-stubs@0.4  openmpi@3.0.1           python@3.5.2
openssl@1automake@1.016.2k1  flex@2.6.3        relion@2 libsigsegv@2.11       openmpi@3.0.3
bwa@0.7.151         freetype@2.7  python@3.6.0
binutils@2.28    libxdmcp@1fltk@1.13.23         openssl@1libtiff@4.0.2n9          relion@2openmpi@3.1
bzip2@1.0.6           freetype@2readline@7.70
binutils@2.128     libxml2@2fltk@1.93.43         pcre@8libtool@2.4.406         openssl@1      relion@2.1
cairo@1.14.8         gcc@6readline@7.30
bison@3.0.4      font-util@1.3.1    libxml2@2libx11@1.96.45          openssl@1 perl@5.24.1              relion@2readline@7.10
cairo@1boost@1.1464.80     gdbm@1fontconfig@2.11.131          libxml2@2.9libxau@1.0.8         perl@5 openssl@1.260.22k          relion@2.0.3
bowtie2@2.3.1   relion@2.1 cmake@3fontconfig@2.712.21  libxcb@1.13    gdbm@1.14.1        libxslt@1openssl@1.10.282k        perl-text-soundex@3.05  relion@2.10.3
cmake@3bwa@0.87.015       gettext@0.19.8.1freetype@2.7       libxslt@1libxdmcp@1.1.292        pixman@0openssl@1.340.0 2n          relion@2.1
cmake@3bzip2@1.90.46      glib@2freetype@2.497.71     libxml2@2.9.4     libxslt@1.1.29    pcre@8.40    pixman@0.34.0           relion@2.1
cmake@3cairo@1.914.48      glib@2gcc@6.533.10          lz4@1libxml2@2.89.1.24           pkg-config@0.29.2perl@5.24.1       relion@3.0_beta cmake@3.9.4      gmp@6relion@2.1.2
cairo@1.14.8     gdbm@1.13          lzma@4libxml2@2.329.78         perl@5.26.2   pkgconf@1.3.10          relion@3relion@2.0_beta1
cmake@3.107.02      help2man@1gdbm@1.4714.41     lzo@2.09   libxslt@1.1.28        perl-text-soundex@3.05  relion@2.1
pkgconf@1cmake@3.48.20        gettext@0.19.8.1   relion@3.0_beta
cmake@3.11.2libxslt@1.1.29        pixman@0.34.0           hmmer@3relion@2.1b21
cmake@3.9.4       m4@1glib@2.4.1849.7        libxslt@1.1.29        py-appdirs@1pixman@0.4.334.0           relion@3.0_beta

Currently, there are 1027 packages (including different versions of the same package) installed. This number will undoubtably increase after the writing of this documentation.

...

How to search the output of: spack find

However, while spack find list the packages alphabetically, it can still be a little difficult to search for packages you are interested in (especially if you don't know its exact/full name). For example, this command doesn't find any packages:

spack find py

However, from the output of the command "spack find" clearly shows numerous packages that start with "py". To improve on Spack's native find command, add this to your ~/.bashrc:

spack_my_find () {
    spack find | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" | grep -E "${1}|-- linux" | sed 's/-- linux/\n-- linux/g'
}

This code provides the command "spack_my_find", which provides a much more robust search of what's installed in Spack; specifically, it allows you to search for any package that contains a given string (regular expressions are supported). For example,

spack_my_find beta

Produces the following output:

-- linux-centos7-x86_64 / gcc@4.8.5 -----------------------------
relion@3.0_beta
relion@3.0_beta
relion@3.0_beta
relion@3.0_beta

-- linux-centos7-x86_64 / gcc@6.3.0 -----------------------------

Getting information about installed packages

In the previous section, we observed that there were 2 installations of python@3.6.0 for rhel6 using gcc version 6.3.0. One way to see the differences between these different installations is the use the following command:

spack find -v -l relion@3.0_beta

In the above command:

• The -v will cause the variants (e.g. compile-time options in package configuration) to be printed. This will allow us to see differences between the 2 packages.
• The -l will cause the package "hashes" to be printed.

Note: a hash is a fixed-length string that is calculated by a hash function, with the software/package of interest as the primary input. If 2 pieces of code have ANY differences, then they will have different hashes. Spack simply uses hashes to, among other things, provide a unique descriptor for each package.

The command produces the following output:

==> 4 installed packages.
relion@2.1
cmake@3.9.4      glib@2.53.1        lz4@1.8.1.2           pkg-config@0.29.2       relion@3.0_beta
cmake@3.9.4      gmp@6.1.2          lzma@4.32.7           pkgconf@1.3.10          relion@3.0_beta
cmake@3.10.0     help2man@1.47.4    lzo@2.09              pkgconf@1.4.2           relion@3.0_beta
cmake@3.11.2     hmmer@3.1b2        m4@1.4.18             py-appdirs@1.4.3        relion@3.0_beta

Currently, there are 1027 packages (including different versions of the same package) installed. This number will undoubtably increase after the writing of this documentation.

Note: the name of each package is given, as well as the version number (e.g.autoconf@2.69). Also notice that the packages are also grouped by the OS and compiler that was used to construct the packages (e.g.linux-rhel6-x86_64 / gcc@4.4.6).

How to search the output of: spack find

However, while spack find list the packages alphabetically, it can still be a little difficult to search for packages you are interested in (especially if you don't know its exact/full name). For example, this command doesn't find any packages:

spack find py

However, from the output of the command spack find clearly shows numerous packages that start with "py". To improve on Spack's native find command, add this to your ~/.bashrc:

function spack_my_find () {
    spack find | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" | grep -E "${1}|-- linux" | sed 's/-- linux/\n-- linux/g'
}

This code provides the command spack_my_find, which provides a much more robust search of what's installed in Spack; specifically, it allows you to search for any package that contains a given string (regular expressions are supported). For example,

spack_my_find beta

Produces the following output:

-- linux-centos7-x86_64 / gcc@4.8.5 -----------------------------
u6dzm4v relion@3.0_beta
build_type=RelWithDebInfo ~cuda cuda_arch= +double~double-gpu+gui purpose=cluster
olcttts relion@3.0_beta
relion@3.0_beta
relion@3.0_beta
build_type=RelWithDebInfo +cuda cuda_arch=60 +double~double-gpu+gui purpose=cluster
f2fjevh relion@3.0_beta build_type=RelWithDebInfo +cuda cuda_arch=60 +double~double-gpu+gui purpose=desktop
opdhbcs relion@3.0_beta build_type=RelWithDebInfo +cuda cuda_arch=70 +double~double-gpu+gui purpose=cluster

First, notice that the unique hashes (the first 7 characters) are given printed in grey, to the left of the each package name. These unique hashes will become important if you wish to use one of these 2 packages

Second, notice that each packages variants are printed with either a "+" or "~", indicating that an option is either included or not. For example, the first package (/u6dzm4v) does not include CUDA (~cuda), whereas the second package (/olcttts) does include CUDA (+cuda). Next, I'll describe how I learn more information about what the various package variants mean for a given package.

More package details!

...

-- linux-centos7-x86_64 / gcc@6.3.0 -----------------------------

Getting information about installed packages

In the previous section, we observed that there were 2 installations of python@3.6.0 for rhel6 using gcc version 6.3.0. One way to see the differences between these different installations is the use the following command:

spack find -v -l relion@3.0_beta

In the above command:

• The -v will cause the variants (e.g. compile-

...

spack info relion

OUTPUT:

...

• time options in package configuration) to be printed. This will allow us to see differences between the 2 packages.
• The -l will cause the package "hashes" to be printed.

Note: a hash is a fixed-length string that is calculated by a hash function, with the software/package of interest as the primary input. If 2 pieces of code have ANY differences, then they will have different hashes. Spack simply uses hashes to, among other things, provide a unique descriptor for each package.

The command produces the following output:

==> 4 installed packages.
-- linux-centos7-x86_64 / gcc@4.8.5 -----------------------------
u6dzm4v relion@3.0_beta build_type=RelWithDebInfo ~cuda cuda_arch= +double~double-gpu+gui purpose=cluster
olcttts relion@3.0_beta build_type=RelWithDebInfo +cuda cuda_arch=60 +double~double-gpu+gui purpose=cluster
f2fjevh relion@3.0_beta build_type=RelWithDebInfo +cuda cuda_arch=60 +double~double-gpu+gui purpose=desktop
opdhbcs relion@3.0_beta build_type=RelWithDebInfo +cuda cuda_arch=70 +double~double-gpu+gui purpose=cluster

First, notice that the unique hashes (the first 7 characters) are given printed in grey, to the left of the each package name. These unique hashes will become important if you wish to use one of these 2 packages

Second, notice that each packages variants are printed with either a + or ~, indicating that an option is either included or not. For example, the first package (/u6dzm4v) does not include CUDA (~cuda), whereas the second package (/olcttts) does include CUDA (+cuda). Next, I'll describe how I learn more information about what the various package variants mean for a given package.

More package details!

In the previous section, we observed that packages can have different variants (e.g. compile-time options). However, spack find -l -v prints abbreviated variant names, which can sometimes be cryptic. For a more complete description of a packages potential variants (as  well as versions/dependencies), use this command (here, we're looking at Relion, but you could enter any package name):

spack info relion

OUTPUT:

CMakePackage:   relion

Description:
    RELION (for REgularised LIkelihood OptimisatioN, pronounce rely-on) is a
    stand-alone computer program that employs an empirical Bayesian approach
    to refinement of (multiple) 3D reconstructions or 2D class averages in
    electron cryo-microscopy (cryo-EM).

Homepage: http://http://www2.mrc-lmb.cam.ac.uk/relion

Tags: 
    None

Preferred version:  
    2.1         [git] https://github.com/3dem/relion.git

Safe versions:  
    develop     [git] https://github.com/3dem/relion.git
    3.0_beta    [git] https://bitbucket.org/scheres/relion-3.0_beta.git
    2.1         [git] https://github.com/3dem/relion.git
    2.0.3       [git] https://github.com/3dem/relion.git

Variants:
    Name [Default]              Profiling,   Allowed values          Description


    build_type [RelWithDebInfo]    Debug, Release,         The build type to build
         Benchmarking                 cuda [on]        RelWithDebInfo,         
    True, False             enable compute on gpu     cuda_arch []        Profiling,           20, 30, 32, 35,
50,     CUDA architecture                             Benchmarking       52, 53, 60, 61, 62, 
    cuda [on]                      True, False           70  enable compute on gpu
    cuda_arch []                double [on]  20, 30, 32, 35, 50,     CUDA architecture
       True, False             double precision (cpu) code     double-gpu [off]     52, 53, 60, 61, 62,     
True, False             double precision gpu     gui [on]             70          True, False           
 build the gui double    purpose [clusteron]              cluster, desktop     True, False  build relion for use in       double precision (cpu) code
    double-gpu [off]               True, False             double precision gpu
    gui [on]       cluster or desktop  Installation Phases:     cmake    build  True, False install  Build Dependencies:     cmake  cuda  fftwbuild the gui
    purpose [cluster]            fltk  libtiffcluster, desktop  mpi  Link Dependencies:   build relion cudafor use in
 fftw  fltk  libtiff  mpi  Run Dependencies:     None  Virtual Packages:      None

This output is pretty self-explanatory. First, the name of the package is given, along with its home page. This is followed by a list of package versions that are currently considered stable by Spack. Next, a description of the specific package variants is given; defaults are shown in square brackets. This is

followed by installation information. Finally, package dependencies are described. These are automatically loaded when the package is loaded.

Listing available packages

Seeing what packages COULD be installed is also very easy! Just use the following command:

spack list

TRUNCATED OUTPUT:

==> 2831 packages.
abinit                                   mustercluster or desktop

Installation Phases:
    cmake    build    install

Build Dependencies:
    cmake  cuda  fftw  fltk r-catools abysslibtiff  mpi

Link Dependencies:
    cuda  fftw  fltk  libtiff  mpi

Run Dependencies:
    None

mvapich2Virtual Packages: 
                            r-cdcfluview
accfft                           mxmlNone

This output is pretty self-explanatory. First, the name of the package is given, along with its home page. This is followed by a list of package versions that are currently considered stable by Spack. Next, a description of the specific package variants is given; defaults are shown in square brackets. This is

followed by installation information. Finally, package dependencies are described. These are automatically loaded when the package is loaded.

Listing available packages

Seeing what packages COULD be installed is also very easy! Just use the following command:

spack list

TRUNCATED OUTPUT:

==> 6768 packages.
abinit                           muster        r-cellranger ack                        r-catools
abyss     mxnet                       mvapich2           r-cghflasso activeharmony                   r-cdcfluview
nagaccfft                           mxml         r-checkmate adept-utils                      nalu   r-cellranger
ack                              mxnet r-checkpoint adios                            nalu-wind    r-cghflasso
activeharmony                    nag     r-chemometrics adios2                           namd   r-checkmate
adept-utils                      nalu         r-chron adlbx                         r-checkpoint
adios  nano                          nalu-wind         r-circlize adol-c                    r-chemometrics
adios2      nanoflann                     namd         r-class aegean                         r-chron
adlbx nanopb                           nano      r-classint albany                           nasm r-circlize
adol-c                           nanoflann                              r-class
aegean                           nanopb                                 r-classint
albany                           nasm                                   r-cli

Currently, there are 6,768 packages available! This number will also certainly increase after the writing of this documentation. 

Note: That the output of spack list is plain text, so searching its output is easier than searching the output of spack find

To search this output, just grep its output:

spack list | grep "^r-"

(where you can replace "^r-" with whatever regular expression you wish)

TRUNCATED OUTPUT:

r-a4
r-a4base
r-a4classif
r-a4core
r-a4preproc
r-a4reporting
r-abadata

...

Using Spack Packages

How to load Spack packages

Say we want to use an installation of python that is managed by Spack. First, let's see what versions/installations of python are currently installed:

spack find python

OUTPUT:

==> 19 installed packages.
-- linux-centos7-x86_64 / gcc@4.8.5 -----------------------------
python@2.7.13  python@2.7.14  python@3.4.3  python@3.5.2  python@3.5.2  python@3.6.0

-- linux-centos7-x86_64 / gcc@6.3.0 -----------------------------
python@2.7.13  python@2.7.14  python@2.7.14  python@2.7.15  python@3.4.3  python@3.6.0  python@3.7.0
python@2.7.13  python@2.7.14  python@2.7.15  python@2.7.15  python@3.5.2  python@3.6.2

Notice that there are multiple installations of python, that differ in version number, compiler, etc. As such, it is always a good idea to use the command spack find to observe if there are multiple installations before just loading a package blindly (otherwise you may load an installation that you do not intend).

To load the installation of python v. 3.7.0 that was compiled with GCC v. 6.3.0, use this command:

spack load -r python@3.7.0^gcc@6.3.0

• The -r instructs Spack to load all modules that python@3.7.0 depends on. This is typically done automatically, but not currently done for perl, python, or R (this is because Spack does not build these packages with RPATH support--this may change in the future).

Executing this command doesn't produce any output, but just to illustrate that python@3.7.0 is indeed loaded (as well as its dependencies), execute the following command:

echo $LOADEDMODULES | sed "s/:/\n/g"   r-cli

Currently, there are 2,831 packages available! This number will also certainly increase after the writing of this documentation. 

Note: That the output of "spack list" is plain text, so searching its output is easier than searching the output of "spack find" 

To search this output, just grep its output:

spack list | grep "^r-"

(where you can replace "^r-" with whatever regular expression you wish)

TRUNCATED OUTPUT:

r-a4
r-a4base
r-a4classif
r-a4core
r-a4preproc
r-a4reporting
r-abadata

...

| sort

Here, we can see that python@3.7.0 is indeed loaded, as are all packages that it depends on.

How to use a unload package

To avoid confusion, as well as conserve memory, you may wish to unload a Spack package (this is especially true in complex workflows). Doing so is very easy:

spack unload python@3.7.0

Let's check and make sure we have really unloaded python@3.7.0:

echo $LOADEDMODULES | sed "s/:/\n/g" | sort

OUTPUT:

Related articles

...