This is an approach to launch Jupyter notebook on a compute node of EEC HPC. Normally you launch Jupyter locally and then open associated web interface on your local machine. This is also possible to do on the HPC, however because the compute nodes of the cluster are mostly accessible via CLI only and are not “exposed” to the external to HPC network, one need to tunnel through the headnodes in order to reach them. The provided set of 2 HTA scripts simplifies the procedure: first script submits the Jupyter job to the queuing system of HPC (slurm) using html forms and ssh command tool (plink.exe from PuTTY). The second script (again using plink) establishes the ssh-tunnel to the target node, where the Jupyter server is running and starts the default browser on the client machine, pointing to the local port brought by the tunnel to the client machine. Since the compute nodes of HPC have multiple CPUs (some up to 32 cores), it is also shown that Jupyter notebook can utilise IPython ipcluster for running notebook codes on parallel threads.
link to scripts on coventry github
Video:
0 Comments.