Category Archives: Connecting to HPC

Built-in SSH Commands in Windows 10

Windows 10 now has native support for SSH. To get it active, go to “Manage optional features” and click on “Add feature” to add it (if it is not there yet).

This command line ssh client can be used instead of PuTTY to connect to HPC terminal.

Once enabled, you can just open command prompt (WinKey+R and type: cmd <Enter>). In the console type “ssh zeus.coventry.ac.uk” or “ssh yourhpcusername@zeus.coventry.ac.uk” if your Windows user name is not the same as HPC user name.

Alex

Connecting to HPC terminal with Google Chrome

Instead of using PuTTY, you can also connect to HPC terminal in Google Chrome using SSH extension, see example below:

Secure Shell extension for Google Chrome!

Terminal emulator and SSH client is now available as a Google Chrome browser extension: run your ssh session to HPC from Chrome!

Secure Shell is an xterm-compatible terminal emulator and stand-alone ssh client for Chrome. It uses Native-Client to connect directly to ssh servers without the need for external proxies.

https://chrome.google.com/webstore/detail/secure-shell-extension/iodihamcpbpeioajjeobimgagajmlibd

Alex

GUI HPC VNC jobs for Mac/Linux Clients

  1. not essential, but useful: create password-less access to zeus, in the terminal of your OSX/Linux do:
ssh-keygen

(just press Enter few times, do not enter any passwords or “secrets”)

Changing username to your HPC login name, do

ssh-copy-id username@zeus1.coventry.ac.uk

(enter HPC password as prompted, this will copy public ssh key created above to your zeus profile, now you can connect to zeus without entering password -> very useful for following steps). Only need to be done ONCE (per client computer)!

——————————————————————————————————————-

Next steps can be scripted, i.e. put code lines into shell script and call it later with different node name or number (node name can be passed as a parameter to the script)

2. Launch VNC server job in the zeus queue, say, on 1 GPU node asking for 1 K80 GPU and exclusive access to the node (remove –exclusive if the whole node is not needed)

 

ssh yourHPClogin@zeus1.coventry.ac.uk sbatch --exclusive -N1 --gres=gpu:K80:1 -t 24:00:00 /share/apps/slurm/vnc.slurm

(omit –gres=gpu:K80:1 if no GPUs needed)

(generally -N X in the above code line means ask for X compute nodes)

Check which node was assigned to your VNC job in the queue:

ssh username@zeus1.coventry.ac.uk squeue -u $USERNAME

Say the result of that is compute node zeus400 assigned to your VNC job (you can always check with “qstat” or “squeue -u $USER” on zeus or on zeus website (show queue)

 

3. Establish tunnel from your Mac to target node (e.g. zeus400) via zeus2.coventry.ac.uk (or zeus1):

ssh -L localhost:5901:zeus400:5901 zeus2.coventry.ac.uk

 

4. Now launch vncviewer on your Mac and connect to localhost:1 (that must bring you to the VNC session at zeus400:1

vncviewer localhost:1

(or any other way you launch it on Macs…)
Alex Pedcenko

02 GUI HPC VNC jobs for Windows Clients

For running GUI jobs on HPC compute node (rather than on login node), you can use my new Windows HTA-scripts from here: aa3025/hpc (coventry.ac.uk). They will also work via University VPN (Anyconnect)

##### optional step – usually already done when your HPC account is created ########

Prior to all the following you need to set your VNC password on the HPC (you can also change your VNC password this way):

  • login via ssh to HPC,
  • issue the command “vncpasswd” to change your VNC password (VNC password can be different to your HPC password, but it makes life easier if it is the same as your HPC password). Use this password for connecting to your VNC sessions in future.
####################################################

You can use these “scripts” to launch VNC (Remote Desktop) job on one of the HPC compute nodes if you need access to GUI (Graphical User Interface)

These scripts allow establishing VNC Desktop session on a compute node of HPC : (see also VIDEO screen cast here)

or here:

0) Download the whole distribution from aa3025/hpc (coventry.ac.uk) You have at least to download 2 folders: “tools” and either “zeus” or “pluto” depending on which HPC you use. You can use git for synchronising the whole distribution:

git clone https://github.coventry.ac.uk/aa3025/hpc.git
Alternatively you can download the zip archive from : HERE

1) Depending which HPC you are about to use, choose a folder zeus or pluto to launch VNC session on required HPC. Before proceeding further  check that HPC has free (idle) compute nodes available (“sinfo” in HPC console or check Zeus website or Pluto website)

2) Launch only one script: 01_START_VNC_Session_….._1_node.hta (by double-clicking on it), a.k.a. “submission app”

3) Fill in your HPC username, password and the rest of info. Hit “Submit” button, the script will then submit the “VNC” job to the HPC queue in order to start “vncserver” process [Remote Desktop session] on the assigned compute node.

4) Wait for your VNC job to start in the queue of HPC (you will see job listing in the newly opened black “plink” console window), note the target node name (it is also presented in the last column of the small table at the bottom of the submission app).

5) Once VNC job has started (“Running” state, “R”) in the black console,  you can close it. Make sure your VNC job is in Running state (also specified by “R” against your job ID in the table at the bottom of the submission app, if it is still in “PD” state (means still “pending”), waiting for resources to be allocated).

6) Once you closed the first (job submission) app and black console window, the second script will start automatically to get the ssh-tunnel running between your PC and the allocated compute node and will connect you to the node’s VNC session — fill in the target node name from the previous step and your HPC password. The VNC screen number is always :1 (you can’t change it).

7) Now, the VNC password window will pop up — enter your VNC password there (theoretically it can be different from your HPC password). If you do not have VNC password, return to the top of this page (“Prior to all of the following…”) and set it up.

8) If all went well, the VNC Remote Desktop windows will be presented. You can toggle it to full screen by pressing F8 on your keyboard (amongst some other options in F8 as well).

By default the VNC job will last for 24 hours and will request 1 whole node, you can select different time and less CPUs than on the whole node or more than one compute node to be allocated.

More than one compute node. If you specify more than one node in VNC job submission app, the VNC Desktop session will start on the first allocated node, but the resources of all other allocated nodes will still be available from your VNC Desktop session.

VNC job termination

When the slurm VNC job expires (the time limit of your job [normally 24 hours if you did not change it] was reached), your VNC Desktop session will be terminated.

However, if you wish to terminate the VNC job before it expires (you do not use it any more and need to free the HPC resources), you can either just kill it from ssh console with “scancel JOBID” command, or by selecting “Logout” in your VNC Desktop session and your slurm job with VNC sessions will terminate.

If you just close the VNC viewer window on your PC –> the VNC Desktop session will remain running until termination time is reached or you cancel your VNC job. Make sure you do not waste HPC resources by having idle VNC sessions’ jobs hanging in the queue!

Alex Pedcenko

SSH Tunnel to the HPC’s compute node from Uni network

For more recent Recommended approach see: http://zeus.coventry.ac.uk/wordpress/?p=1119

 

 


If you need to launch GUI application on the actual compute node the job was assigned to by slurm, you can do VNC-session through ssh-tunnel (tunneling from outside to the node via login node).

You can connect “directly” to one of the reserved nodes (node assigned by slurm to your job) from university network (or from outside via VPN) using ssh tunnel. E.g. on your local machine issue the command (we presume SSH is installed on your PC, if not, you can use cygwin from w:\Students\Shared\EC\STUDENT\HPC\cygwin\)

ssh -L 5901:zeus15:5901 username@zeus4.coventry.ac.uk

This will connect the port 5901 (i.e. VNC-server session :1 running on zeus15) to the port 5901 (you can hook it to any other if 5901 is already taken on your PC) of your local machine via login node zeus4. Obviously replace “username” by your HPC user name. Then, to connect to VNC session running on zeus15:1, just connect your vncviewer to localhost:1. If you need to forward any other non-privileged port from compute node to your local PC, just change the port numbers in the command above.

vncviewer localhost:1

01 Connecting to HPC

OK, you’ve got an HPC account, what next?

If you have got the HPC account (on either zeus or pluto HPC), the first thing you need to do is to change your temporary password. For this, you need to log in to HPC terminal/console with your temporary password and issue the command passwd.

This can be done by establishing SSH connection with one of the HPC headnodes (either zeus.coventry.ac.uk or pluto.coventry.ac.uk depending on which HPC you have the account on). SSH is a secure shell protocol used to login to remote computers via command line. There are few ssh-client apps allowing you to do this.

Connecting in console with secure shell (ssh)

To do that you will need ssh-client software. The possibilities are:

  • Windows “native” ssh client (command line). If you have relatively recent copy of Windows 10, just open command window (Winkey+R: cmd.exe, Enter)
  • If you are using Linux or Mac computer you already have native ssh client.

In either of these cases just open a terminal and type

ssh yourhpcusername@zeus.coventry.ac.uk

or if you are using pluto HPC:

ssh yourhpcusername@pluto.coventry.ac.uk

Using PuTTY as ssh-client

PuTTY free ssh client (also available from AppsAnywhere https://appsanywhere.coventry.ac.uk), or can be downloaded and installed on your desktop PC e.g. from https://www.chiark.greenend.org.uk/~sgtatham/putty/. The documentation for using PuTTY can be found at http://www.chiark.greenend.org.uk/~sgtatham/putty/docs.html

Using Google Chrome as ssh-client

You can install ssh extension for both Google Chrome or MS Edge browsers:

Secure Shell – Chrome Web Store (google.com)

Using Windows 10 Linux subsystem (WSL) apps

Since recently you can install mini Linux distribution on your Windows 10 PC. What you need is to first install optional Windows component (Control Panel -> Programs -> Turn Windows Features on/off -> Windows Subsystem for Linux (WSL)). Once that done, reboot your PC, then go to Windows App store and get some Linux distro you are familiar with: Ubuntu, Debian etc and install it. Launch it, and you will be brought to full-functional Linux terminal, where you can install standard Linux console applications (for sure you may need “openssh-client” (Ubuntu) or “ssh” (Debian), e.g. for Ubuntu do “apt update; apt install openssh-client”). It is usefull to set up your WSL username the same as on HPC (saves time entering it all the time upon each ssh connection).

Connecting to the Desktop of HPC with VNC session (Remote Desktop like)

If you want to have a “Desktop” experience on HPC, once you have changed your temporary HPC password, you can use this way for VNC jobs/connection, see here : VNC jobs GUI HPC VNC jobs for Windows Clients | EEC High Performance Computing (coventry.domains)


Alex Pedcenko

Openconnect-gui –opensource free Windows x86/64 alternative for Cisco anyconnect VPN Client

If you are using windows (especially Windows 10 [update Sept-2015: now Cisco works with Win 10]), you may find that Cisco Openconnect VPN client for accessing University network from the outside is crashing on creating the connection. You can use opensource alternative openconnect:

 

https://github.com/openconnect/openconnect-gui/wiki

 

To connect use:

server: connect.coventry.ac.uk
username: your Uni one
Group: VPN
password: your Uni one

Alex

Windows HPC Connection Starter Pack

Is available from W:/EC/STUDENT/hpc/

Includes:

All there programms are free and can be found and installed from web. PuTTY and TigerVNC are also available from AppsAnywhere.

Alex Pedcenko

VPN from Linux Home PC to CU network (i.e access to license servers or HPCs)

To connect to campus network you have to install openconnect — the Open source version of the for Cisco VPN AnyConnect Client.

for Ubuntu /Debian I do:

apt-get install openconnect

or for RedHat/Fedora

yum install openconnect

Once installed do in terminal:

sudo openconnect --user=ax3333@coventry.ac.uk --no-dtls --authgroup='VPN' --no-cert-check anyconnect.coventry.ac.uk

where ax3333@coventry.ac.uk is your user name on CU network. You will be asked for Password (your University one), enter it.

Leave this terminal window running in a background, There are also OpenConnect plug-ins for NetworkManager, but CLI method described here is much simpler (AND WORKS!).
Then use CU Intranet or whatever (e.g. access license servers in CU private network).

GUI method (will not require sudo privileges to setup):

Install OpenConnect add-on for Network Manager (https://launchpad.net/ubuntu/+source/network-manager-openconnect) for Ubuntu:

apt install network-manager-openconnect
or
yum install NetworkManager-openconnect
for RedHat/CentOS.

Then you can add new VPN connection via NetworkManager GUI tool, similarly VPN server must be “anyconnect.coventry.ac.uk” , Group “VPN” (default), “accept all certificates”, use your Uni usrrname/password to connect to VPN.

Alex

css.php