Jupyter Notebooks

1 Installation
- 1.1 SCU - BRB Cluster
  - 1.1.1 Reserve session on node
  - 1.1.2 Perform jupyter-notebook installation
2 Connecting to Jupyter Notebook
- 2.1 Prerequisite
  - 2.1.1 Configure Local SSH Config File
- 2.2 Run Jupyter Notebook
- 2.3 SCU - Cluster Connection
3 Contact

Installation

SCU - BRB Cluster

To connect to the SCU - BRB cluster, you must be connected to the WCM VPN service

Reserve session on node

Jupyter-notebook can be installed from within a node. To reserve an interactive session on a node, the following commands can be run:

[user@my_workstation ~]$ ssh my_cwid@scu-login02.med.cornell.edu
[cwid@scu-login02 ~]$ srun --pty -n1 --mem=8G -p scu-cpu /bin/bash -i

More slurm information can be found here:

Perform jupyter-notebook installation

[rma3001@scu-node001 ~]$ conda create -n jupyter-notebook
[rma3001@scu-node001 ~]$ conda activate jupyter-notebook
[rma3001@scu-node001 ~]$ conda install -c anaconda jupyter

Connecting to Jupyter Notebook

Prerequisite

Configure Local SSH Config File

Configure your ~/.ssh/config file on your local workstation to properly connect to the internal node that will run your Jupyter Notebook:

Replace MY_CWID, with your CWID:

Host scu-node*
        user MY_CWID
        ProxyCommand ssh -W %h:%p scu-login02
        
Host scu-login02
        Hostname scu-login02.med.cornell.edu
        user MY_CWID
        ServerAliveInterval 60
        TCPKeepAlive yes

For seamless access to the cluster and Jupyter Notebooks, ensure your SSH Keys are configured

Run Jupyter Notebook

BRB
Jupyter notebook can be started through the following command:

Keep track of the http://127.0.0.1:port/?token link. This will be needed once the SSH connection is established.

For Curie (Greenberg) change --port=8959

From another terminal

ssh -o ProxyCommand="ssh -W %h:%p userid@aphrodite.med.cornell.edu" -L localhost:8959:node159.panda.pbtech:8959userid@node159.panda.pbtech

SCU - Cluster Connection

From within a different terminal, edit the ~/.ssh/config file on your local workstation to include the following, replace MY_CWID, with your CWID:

Once this is configured, you can run the following command on your local terminal, replace “scu-nodeXYZ” with the node that is running the Jupyter notebook

As expected, this command will hang and will not produce any output.

On your local machine’s browser, enter the token link that was outputted when Jupyter notebook was run, beginning with http://127.0.0.1:port/?token=…

Contact

Contact scu@med.cornell.edu if you have any questions or if any issues are encountered.