Modules
- eud4002
- Meghan Netterville
What are Environment Modules?
Environment modules are a powerful system for managing software packages and their dependencies in high-performance computing (HPC) environments. They provide a flexible way to dynamically modify a user's shell environment, allowing easy access to different software versions without conflicts.
SCU team uses modules on all of the HPC clusters, so this document applies to any of them. But availability and naming of certain modules varies between different HPC systems.
How Modules work
When you load a module, it dynamically updates several critical environment variables, such as:
PATH: Adds the module's binary directories
LD_LIBRARY_PATH: Includes the module's library directories
MANPATH: Adds module-specific manual page locations
PYTHONPATH: Extends Python module search paths
PKG_CONFIG_PATH: Updates package configuration paths
Bash (and other shells) use these variables to determine where to find executables, shared libs, manpages, and so on. Here is an illustration of what happens to the PATH variable when a module is loaded and unloaded:
# Before loading any module python executable isn't available
$ which python
$
$ echo $PATH
/usr/local/bin:/usr/bin:/bin
# Load a specific Python module (anaconda python distribution in this example)
$ module load anaconda3
# Now Python points to the module's version
$ which python
/shared-apps/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-13.2.0/anaconda3-2022.10-34zllqwvviae6dpyqsumcgnrqrvwykaf/bin/python
$ echo $PATH
/shared-apps/spack/opt/spack/linux-ubuntu22.04-x86_64/gcc-13.2.0/anaconda3-2022.10-34zllqwvviae6dpyqsumcgnrqrvwykaf/bin:/usr/local/bin:/usr/bin:/bin
# note how the PATH variable has an anaconda3 installation path in the list
# Unload the module
$ module unload python
# Back to the system default, where no python is available in the PATH
$ which python
$This illustrates how you can load the module and then simply call python instead of using an absolute path to it, which can be very long and cumbersome to type.
Basic Module Commands
Listing Available Modules
# List all available modules
module avail
# Search for specific modules
module avail python
module avail gccLoading and Unloading Modules
# Load a specific module
module load python/3.9
# Load a specific version of a module
module load cuda/11.3
# Unload a module
module unload python
# swap versions of the application
module swap python/3.9 python/3.10
# Completely reset your environment
module purge
# get help
module helpChecking Module Information
# Show details about a specific module
module show python/3.9
# List currently loaded modules
module listYou may need to include module load <application_name> command in the SLURM script so that bash can find path to application binaries