How do I use docker on discovery?

Discovery Questions
1

I have a docker container containing my software stack for a research project. How can I use this on discovery?

2 Likes
2

Unfortunately, you can’t – at least, not directly. Docker isn’t available on Discovery (nor almost any HPC cluster) because the Docker daemon requires root permissions to run, which would open up some major security risks.

Fortunately, Singularity was created essentially for this reason, and is installed on Discovery! You can convert your Docker container to a Singularity container fairly easily:

  1. Save and gzip the local Docker image you use to run your container

    docker save your_image | gzip > your_image.tar.gz

  2. Copy the image file to Discovery

    scp your_image.tar.gz username@discovery7.hpcc.dartmouth.edu:/remote/path/your_image.tar.gz

  3. On Discovery, build a Singularity image from the file

    singularity build image_name docker-archive://your_image.tar.gz

You can then execute commands inside a Singularity container with singularity exec image_name some_command or launch an interactive shell with singularity shell image_name.

Other notes:

  • If you have a Docker Hub account, you can also docker push your local image to Docker Hub and build the Singularity container with singularity build image_name docker://username/repository rather than saving and scp'ing the tarball.

  • By default, Singularity containers are read-only, meaning you can’t create, edit, or delete any files inside them one built. Normally, you could create a writable container by building it with the --sandbox or --writable flags, but those require root permissions and don’t work on Discovery. If you want to be able to create or modify files from inside your container, you can get around this by creating a directory on Discovery for them, and bind-mounting it to the container by passing --bind discovery/dir/path/:/mnt/ to the exec or shell commands (you only have to do this once).

  • Bind-mounting is super useful in general when running Singularity on Discovery and can help make containers way smaller and faster. For example, say your containerized code runs fMRIPrep on a dataset. Rather than having your Dockerfile ADD/COPY a bunch of NIfTI files, you could create a read-only bind-mount the data (--bind discovery/data/dir:/container/data/dir:ro). And rather than installing a heavy library like FSL, you could bind-mount /dartfs-hpc/admin/opt/el7/fsl/5.0.10/ (or a different version listed by module avail) and export PATH=/dartfs-hpc/admin/opt/el7/fsl/5.0.10/bin:$PATH

1 Like
3

Great answer @paxton!

In our lab, @ejolly has built a shared singularity container that has all of the imaging software we use. However, in order to allow us to easily maintain our own python modules, we create a unique environment for each user and bind this to the container.

Here is the script we run to launch the container:

#!/bin/bash
ME=`whoami`
MYDIR="${ME: -1}"
#module load singularity/2.4.2
singularity run \
	-c \
	-e \
	-B /dartfs/rc/lab/D/DBIC/cosanlab/datax/Resources/cosantools/container_pkgs:/container_pkgs \
	-B /dartfs:/dartfs \
	-B /idata:/idata \
	-B /scratch:/tmp \
	-B /scratch:/var \
	-B /dartfs-hpc/rc/home/$MYDIR/$ME:/dartfs-hpc/rc/home/$MYDIR/$ME \
	/dartfs/rc/lab/D/DBIC/cosanlab/datax/Resources/cosantools/cosanTools.img "$@"
1 Like
4

Just a note that the deployed version that side loads python environments is on the “dev” branch of the repo that @ljchang linked to.