Andy Connolly wrote up an example in the DBIC Handbook. However, this is unfortunately now outdated since the migration to the SLURM scheduler. Follow his instructions on how to create a password for your jupyter notebook servers.
Here is an updated job submission script. Note that this script assumes that you have a python distribution available in your shell environment. For example, I maintain my own conda distribution. However, if you would prefer to use the research computing distribution, then load their module by uncommenting the line module load jupyter
.
#!/bin/bash -l
#SBATCH --job-name=JupyterNotebook
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --time=10:00:00
# get tunneling info
XDG_RUNTIME_DIR=""
node=$(hostname -s)
user=$(whoami)
cluster="discovery7"
# This next command chooses a random port number between 8000 and 8999
port=`echo $(( 8000 + RANDOM % 1000 ))`
# print tunneling instructions jupyter-log
echo -e "
# Command to create ssh tunnel:
ssh -N -f -L ${port}:${node}:${port} ${user}@${cluster}.dartmouth.edu
# Use a Browser on your local machine to go to:
localhost:${port}
"
# Uncomment the following line to load Python Module, Otherwise this assumes you are using your own python distribution
# module load python
jupyter-notebook --no-browser --port=${port} --ip=${node}
# keep it up and running
sleep 3600
now submit job to the cluster.
(base) [f00275v@discovery7 code]$ sbatch jupyter_notebook.pbs
Submitted batch job 1295256
There will be a new text file generated with the unique job id containing instructions on how to tunnel into your server. Let’s read this file:
(base) [f00275v@discovery7 code]$ cat slurm-1295256.out
# Command to create ssh tunnel:
ssh -N -f -L 8644:j12:8644 f00275v@discovery7.dartmouth.edu
# Use a Browser on your local machine to go to:
localhost:8644
Now open a terminal on your local machine and paste in the command to create the ssh tunnel:
(base) lukechang@Lukes-MacBook-Air ~ % ssh -N -f -L 8644:j12:8644 f00275v@discovery.dartmouth.edu
Open a web browser on your local machine and type localhost:8254
. You will now be prompted for the password you generated using jupyter notebook password
If you would like to view the status of your jobs, use squeue
.
(base) [f00275v@discovery7 code]$ squeue
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON)
1295256 standard JupyterN f00275v R 18:06 1 j12
if you would like to kill your job running the jupyter server when you are done, use scancel
(base) [f00275v@discovery7 code]$ scancel 1295256