Running Docker images on Singularity

Singularity is a container framework for Linux. Unlike Docker, it can be utilized on the HPC cluster. This guide shows the basics of converting existing docker containers to singularity containers. To learn more about Singularity and Docker, see the official documentation.

Accessing singularity

You need to set up Spack first. Once that is done, here is how you access Singularity in an interactive session:

# Log in to HPC submit note
ssh curie.pbtech

# Start interactive HPC session
srun -n1 --pty --partition=panda --mem=8G bash -i

# Load singularity (NB: Spack setup must be completed)
spack load singularity@2.6.0

# Verify singularity is available
singularity --version
> 2.6.0-dist

Building singularity images from docker images

In this example, we’ll build the “lolcow” test image from the public docker hub, and run it:

# Note: The `docker://` prefix is required
singularity pull docker://godlovedc/lolcow
> [..]
> Done. Container is at: ./lolcow.simg

singularity run lolcow.simg
>  __________________
> < Hello, world! >
>  ------------------
>         \   ^__^
>          \  (oo)\_______
>             (__)\       )\/\
>                 ||----w |
>                 ||     ||

Authentication with private docker DTR

To access a private docker DTR, it is advisable to create an access token first. With that, do the following:

# Setup token file
echo your-token-goes-here > dtr.token
chmod 600 dtr.token

# Setup environment. DTR username is case sensitive
export SINGULARITY_DOCKER_USERNAME=your_dtr_username
export SINGULARITY_DOCKER_PASSWORD=$(cat dtr.token)

# Now, you can pull from your private DTR
# Again, the `docker://` prefix is required
singularity pull docker://dtr.example.com/group/image_name

Please note that these environment variables will be cleared once you end the interactive HPC session (via exit).

Mounting host volumes

In docker, you can bind host volumes during runtime with the -v flag. In Singularity, the equivalent flag is -B:

This will bind the folder /host/one to /one within the container, and the file /host/two/file.txt to /file.txt within the container.

For more details, see the Singularity documentation on Bind Paths and Mounts.

Environment variables

In Docker, you can pass environment variables during runtime with the --env-file flag.

Singularity, by default, mirrors the host environment, so you can set the environment there. Alternatively, you can set environment variables by prepending them with SINGULARITYENV. In this example, we set SINGULARITYENV_HELLO, which is accessible within the container as HELLO:

This will overwrite any host environment variables with the same name. If you need to clear the host environment data, use --cleanenv.

For more details, see the Singularity documentation on Environment and Metadata.