| Table of Contents | ||
|---|---|---|
|
...
Spack Introduction and Environment Setup
What is Spack?
Spack is a package manager, similar to yum, apt-get, or conda. It allows for multiple package versions/configurations. Unlike the other package managers listed, Spack is specifically designed to be used on HPC resources with varying architectures.
The purpose of this documentation is to provide a practical guide to quickly enable you to take advantage of its use in your research.
...
Next, add the following code to your ~/.bashrc, which will automatically load Spack into your shell on all servers besides gateways:
| Code Block | ||
|---|---|---|
| ||
# detect kernel version, then source spack file if it exists _KERNEL_VERSION=$(uname -r | cut -d '.' -f1,2,3) if [[ "${HOSTNAME}" != "pascal.med.cornell.edu" ]] || [[ "${HOSTNAME}" != "aphrodite.med.cornell.edu" ]] || [[ "${HOSTNAME}" != "aristotle.med.cornell.edu" ]]; then if if [[ "${_KERNEL_VERSION}" == *"2.6.3"* ]] ; then if [ if [ -f /softlib/apps/EL6/spack/share/spack/setup-env.sh ] ; then . /softlib/apps/EL6/spack/share/spack/setup-env.sh fi fi if fi fi if [[ "${_KERNEL_VERSION}" == *"3.10"* ]] ; then if if [ -f /softlibsoftware/appsspack/EL7/spackcentos7/share/spack/setup-env.sh ] ; then . /softlibsoftware/appsspack/EL7/spackcentos7/share/spack/setup-env.sh fi fi fi |
| Warning |
|---|
As always, after editing your ~/.bashrc, execute the command |
...
elif [ -f /software/spack/share/spack/setup-env.sh ] ; then
. /software/spack/share/spack/setup-env.sh
fi
fi
fi
builtin unset _KERNEL_VERSION |
| Warning |
|---|
As always, after editing your ~/.bashrc, execute the command |
...
Package Information
How to show what packages are installed in Spack
...
However, from the output of the command spack find clearly shows numerous packages that start with "py". To improve on Spack's native find command, add this to your ~/.bashrc:
| Code Block |
|---|
function spack_my_find () {
spack find | sed -r "s/\x1B\[([0-9]{1,2}(;[0-9]{1,2})?)?[m|K]//g" | grep -E "${1}|-- linux" | sed 's/-- linux/\n-- linux/g'
} |
...
| Warning |
|---|
Note: Sometimes, you'll see multiple installations of the same package installed on the same OS, with the same compiler (e.g. relion@3.0_beta is listed 3 times in the above output). What this means is that this packages was installed for the same OS/compiler, but with different configuration options. This will be explained in the next section. |
...
TRUNCATED OUTPUT:
| Code Block |
|---|
==> 28316768 packages. abinit muster r-catools abyss mvapich2 r-cdcfluview accfft mxml r-cellranger ack mxnet r-cghflasso activeharmony nag r-checkmate adept-utils nalu r-checkpoint adios nalu-wind r-chemometrics adios2 namd r-chron adlbx nano r-circlize adol-c nanoflann r-class aegean nanopb r-classint albany nasm r-cli |
Currently, there are 2are 6,831 768 packages available! This number will also certainly increase after the writing of this documentation.
...
| Code Block |
|---|
==> 19 installed packages. -- linux-centos7-x86_64 / gcc@4.8.5 ----------------------------- python@2.7.13 python@2.7.14 python@3.4.3 python@3.5.2 python@3.5.2 python@3.6.0 -- linux-centos7-x86_64 / gcc@6.3.0 ------------------------------ python@2.7.13 python@2.7.14 python@2.7.14 python@2.7.15 python@3.4.3 python@3.6.0 python@3.7.0 python@2.7.13 python@2.7.14 python@2.7.15 python@2.7.15 python@3.5.2 python@3.6.2 |
Notice that there are multiple installations of python, that differ in version number, compiler, etc. As such, it is always a good idea to use the command spack find to observe if there are multiple installations before just loading a package blindly (otherwise you may load an installation that you do not intend).
| Warning |
|---|
Before loading any Spack packages, it is helpful to know that Spack stores a list of loaded packages in the variable |
To load the installation of python v. 3.7.0 that was compiled with GCC v. 6.3.0, use this command:
| Code Block |
|---|
spack load -r python@3.7.0^gcc@6.3.0 |
- The -r instructs Spack to load all modules that python@3.7.0 depends on. This is typically done automatically, but not currently done for perl, python, or R (this is because Spack does not build these packages with RPATH support--this may change in the future).
...
python@2.7.13 python@2.7.14 python@2.7.14 python@2.7.15 python@3.4.3 python@3.6.0 python@3.7.0
python@2.7.13 python@2.7.14 python@2.7.15 python@2.7.15 python@3.5.2 python@3.6.2 |
Notice that there are multiple installations of python, that differ in version number, compiler, etc. As such, it is always a good idea to use the command spack find to observe if there are multiple installations before just loading a package blindly (otherwise you may load an installation that you do not intend).
| Warning |
|---|
Before loading any Spack packages, it is helpful to know that Spack stores a list of loaded packages in the variable |
To load the installation of python v. 3.7.0 that was compiled with GCC v. 6.3.0, use this command:
| Code Block |
|---|
spack load -r python@3.7.0^gcc@6.3.0 |
- The -r instructs Spack to load all modules that python@3.7.0 depends on. This is typically done automatically, but not currently done for perl, python, or R (this is because Spack does not build these packages with RPATH support--this may change in the future).
Executing this command doesn't produce any output, but just to illustrate that python@3.7.0 is indeed loaded (as well as its dependencies), execute the following command:
| Code Block |
|---|
echo $LOADEDMODULES | sed "s/:/\n/g" | sort |
Here, we can see that python@3.7.0 is indeed loaded, as are all packages that it depends on.
How to use a unload package
To avoid confusion, as well as conserve memory, you may wish to unload a Spack package (this is especially true in complex workflows). Doing so is very easy:
| Code Block |
|---|
spack unload python@3.7.0 |
Let's check and make sure we have really unloaded python@3.7.0:
| Code Block |
|---|
echo $LOADEDMODULES | sed "s/:/\n/g" | sort |
Here, we can see that python@3.4.3 is indeed loaded, as are all packages that it depends on.
...
OUTPUT:
Related articles
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...