Singularity is a container framework for Linux. Unlike Docker, it can be utilized on the HPC cluster. This guide shows the basics of converting existing docker containers to singularity containers. To learn more about Singularity and Docker, see the official documentation.
Accessing singularity
You need to set up Spack first. Once that is done, here is how you access Singularity in an interactive session:
# Log in to HPC submit note ssh curie.pbtech # Start interactive HPC session srun -n1 --pty --partition=panda --mem=8G bash -i # Load singularity (NB: Spack setup must be completed) spack load singularity@2.6.0 # Verify singularity is available singularity --version > 2.6.0-dist
Building singularity images from docker images
In this example, we’ll build the “lolcow” test image from the public docker hub, and run it:
# Note: The `docker://` prefix is required singularity pull docker://godlovedc/lolcow > [..] > Done. Container is at: ./lolcow.simg singularity run lolcow.simg > __________________ > < Hello, world! > > ------------------ > \ ^__^ > \ (oo)\_______ > (__)\ )\/\ > ||----w | > || ||
Authentication with private docker DTR
To access a private docker DTR, it is advisable to create an access token first. With that, do the following:
# Setup token file echo your-token-goes-here > dtr.token chmod 600 dtr.token # Setup environment. DTR username is case sensitive export SINGULARITY_DOCKER_USERNAME=your_dtr_username export SINGULARITY_DOCKER_PASSWORD=$(cat dtr.token) # Now, you can pull from your private DTR # Again, the `docker://` prefix is required singularity pull docker://dtr.example.com/group/image_name
Please note that these environment variables will be cleared once you end the interactive HPC session (via exit).
Mounting host volumes
In docker, you can bind host volumes during runtime with the -v flag. In Singularity, the equivalent flag is -B:
singularity -B /host/one:/one,/host/two/file.txt:/file.txt example.simg
This will bind the folder /host/one to /one within the container, and the file /host/two/file.txt to /file.txt within the container.
For more details, see the Singularity documentation on Bind Paths and Mounts.
Environment variables
In Docker, you can pass environment variables during runtime with the --env-file flag.
Singularity, by default, mirrors the host environment, so you can set the environment there. Alternatively, you can set environment variables by prepending them with SINGULARITYENV. In this example, we set SINGULARITYENV_HELLO, which is accessible within the container as HELLO:
SINGULARITYENV_HELLO=world singularity exec centos7.img env | grep HELLO > HELLO=world
If you need to clear the host environment data, use --cleanenv.
For more details, see the Singularity documentation on Environment and Metadata.