OpenMPI v.4.0.0 is available on zeus HPC. Two versions, compiled with intel compiler 2019:
module load openmpi/4.0.0/intel2019
and gcc-7.1.0
module load /openmpi/4.0.0/gcc7.1.0
There is also openmpi-2.1.5 compiled with gcc-7.1.0
module load openmpi/2.1.5/gcc7.1.0
Alex
There is only one queue/partition named “all” (default) on zeus HPC. For more details see this post:
Queues of zeus [updated June 2018] | EEC High Performance Computing (coventry.domains)
Alex Pedcenko
Terminal emulator and SSH client is now available as a Google Chrome browser extension: run your ssh session to HPC from Chrome!
Secure Shell is an xterm-compatible terminal emulator and stand-alone ssh client for Chrome. It uses Native-Client to connect directly to ssh servers without the need for external proxies.
https://chrome.google.com/webstore/detail/secure-shell-extension/iodihamcpbpeioajjeobimgagajmlibd
Alex
ssh-keygen
(just press Enter few times, do not enter any passwords or “secrets”)
Changing username to your HPC login name, do
ssh-copy-id username@zeus1.coventry.ac.uk
(enter HPC password as prompted, this will copy public ssh key created above to your zeus profile, now you can connect to zeus without entering password -> very useful for following steps). Only need to be done ONCE (per client computer)!
——————————————————————————————————————-
Next steps can be scripted, i.e. put code lines into shell script and call it later with different node name or number (node name can be passed as a parameter to the script)
2. Launch VNC server job in the zeus queue, say, on 1 GPU node asking for 1 K80 GPU and exclusive access to the node (remove –exclusive if the whole node is not needed)
ssh yourHPClogin@zeus1.coventry.ac.uk sbatch --exclusive -N1 --gres=gpu:K80:1 -t 24:00:00 /share/apps/slurm/vnc.slurm
(omit –gres=gpu:K80:1 if no GPUs needed)
(generally -N X in the above code line means ask for X compute nodes)
Check which node was assigned to your VNC job in the queue:
ssh username@zeus1.coventry.ac.uk squeue -u $USERNAME
Say the result of that is compute node zeus400 assigned to your VNC job (you can always check with “qstat” or “squeue -u $USER” on zeus or on zeus website (show queue)
3. Establish tunnel from your Mac to target node (e.g. zeus400) via zeus2.coventry.ac.uk (or zeus1):
ssh -L localhost:5901:zeus400:5901 zeus2.coventry.ac.uk
4. Now launch vncviewer on your Mac and connect to localhost:1 (that must bring you to the VNC session at zeus400:1
vncviewer localhost:1
(or any other way you launch it on Macs…)
Alex Pedcenko
For running GUI jobs on HPC compute node (rather than on login node), you can use my new Windows HTA-scripts from here: aa3025/hpc (coventry.ac.uk). They will also work via University VPN (Anyconnect)
##### optional step – usually already done when your HPC account is created ########
Prior to all the following you need to set your VNC password on the HPC (you can also change your VNC password this way):
You can use these “scripts” to launch VNC (Remote Desktop) job on one of the HPC compute nodes if you need access to GUI (Graphical User Interface)
These scripts allow establishing VNC Desktop session on a compute node of HPC : (see also VIDEO screen cast here)
or here:
0) Download the whole distribution from aa3025/hpc (coventry.ac.uk) You have at least to download 2 folders: “tools” and either “zeus” or “pluto” depending on which HPC you use. You can use git for synchronising the whole distribution:
git clone https://github.coventry.ac.uk/aa3025/hpc.git
1) Depending which HPC you are about to use, choose a folder zeus or pluto to launch VNC session on required HPC. Before proceeding further check that HPC has free (idle) compute nodes available (“sinfo” in HPC console or check Zeus website or Pluto website)
2) Launch only one script: 01_START_VNC_Session_….._1_node.hta (by double-clicking on it), a.k.a. “submission app”
3) Fill in your HPC username, password and the rest of info. Hit “Submit” button, the script will then submit the “VNC” job to the HPC queue in order to start “vncserver” process [Remote Desktop session] on the assigned compute node.
4) Wait for your VNC job to start in the queue of HPC (you will see job listing in the newly opened black “plink” console window), note the target node name (it is also presented in the last column of the small table at the bottom of the submission app).
5) Once VNC job has started (“Running” state, “R”) in the black console, you can close it. Make sure your VNC job is in Running state (also specified by “R” against your job ID in the table at the bottom of the submission app, if it is still in “PD” state (means still “pending”), waiting for resources to be allocated).
6) Once you closed the first (job submission) app and black console window, the second script will start automatically to get the ssh-tunnel running between your PC and the allocated compute node and will connect you to the node’s VNC session — fill in the target node name from the previous step and your HPC password. The VNC screen number is always :1 (you can’t change it).
7) Now, the VNC password window will pop up — enter your VNC password there (theoretically it can be different from your HPC password). If you do not have VNC password, return to the top of this page (“Prior to all of the following…”) and set it up.
8) If all went well, the VNC Remote Desktop windows will be presented. You can toggle it to full screen by pressing F8 on your keyboard (amongst some other options in F8 as well).
By default the VNC job will last for 24 hours and will request 1 whole node, you can select different time and less CPUs than on the whole node or more than one compute node to be allocated.
More than one compute node. If you specify more than one node in VNC job submission app, the VNC Desktop session will start on the first allocated node, but the resources of all other allocated nodes will still be available from your VNC Desktop session.
When the slurm VNC job expires (the time limit of your job [normally 24 hours if you did not change it] was reached), your VNC Desktop session will be terminated.
However, if you wish to terminate the VNC job before it expires (you do not use it any more and need to free the HPC resources), you can either just kill it from ssh console with “scancel JOBID” command, or by selecting “Logout” in your VNC Desktop session and your slurm job with VNC sessions will terminate.
If you just close the VNC viewer window on your PC –> the VNC Desktop session will remain running until termination time is reached or you cancel your VNC job. Make sure you do not waste HPC resources by having idle VNC sessions’ jobs hanging in the queue!
Alex Pedcenko
Queuing model UPDATE June 2018
To simplify usage of different queues, we combined all nodes into a single default queue (slurm partition) “all”. The usage limits are now solely user-based, each user has default (for now) number of CPU*minutes that they can use at any time moment (subject to available resources). If this number of CPU*min is reached, the new jobs from this user will be put on queue until their running jobs will free resources. This is independent of type of compute nodes. During this initial stage we will try to adapt the default CPU*min allowance to suite better and more effective HPC usage. The simple principle behind this is that user can use more CPU cores but for less time, or, less CPU cores, but for longer time. The run time of the job to be submitted is determined by the value you set in –time or -t parameter during the submission of a job (e.g. -t 24:00:00).
If you require a particular type of compute nodes (CPU/GPU/Phi etc), this can be done in submission script or during the submission with sbatch command: by specifying an additional parameter “constraint”:
--constraint=broadwell--constraint=nehalem--constraint=sandy--constraint=smp--gres=gpu:K20:N (where N is the Nr of GPUs needed, max is 2 GPUs/node)--gres=gpu:K80:N (where N is the Nr of GPUs needed, max is 2 GPUs/node)--gres=mic:N or --constraint=phi.For more details on Zeus’s CPUs and nodes see this post: http://zeus.coventry.ac.uk/wordpress/?p=336
If you have no particular preference on the type of CPU or compute node and are running parallel job, please specify ONLY TOTAL Nr of CPUs required, NOT Nr of nodes!: SLURM will assign the nodes automatically.
e.g. if I need 64 CPUs in total for 24 hours on whatever available nodes I submit my slurm script with:
sbatch -n 64 -t 24:00:00 myslurmscriptname.slurm
if I need 64 CPUs in total for 48 hours on Broadwell-based nodes (32 CPUs/node) I submit my slurm script with:
sbatch -n 64 -t 48:00:00 --constraint=broadwell myslurmscriptname.slurm
Finally if I want 2 GPU nodes with 2 Nvidia Kepler K80 GPUs (1 GPUs/node) and 2 CPUs on each node (36 hours), I do something like:
sbatch -N2 --ntasks-per-node=1 --cpus-per-task=2 --gres=gpu:K80:1 -t 36:00:00 mygpuscript.slurm
Certainly some variations of these sbatch commands are possible, also these flags can be specified inside the slurm submission script itself. For full list of possible sbatch options see slurm docs: https://slurm.schedmd.com/sbatch.html
Alex Pedcenko
For more recent Recommended approach see: http://zeus.coventry.ac.uk/wordpress/?p=1119
If you need to launch GUI application on the actual compute node the job was assigned to by slurm, you can do VNC-session through ssh-tunnel (tunneling from outside to the node via login node).
You can connect “directly” to one of the reserved nodes (node assigned by slurm to your job) from university network (or from outside via VPN) using ssh tunnel. E.g. on your local machine issue the command (we presume SSH is installed on your PC, if not, you can use cygwin from w:\Students\Shared\EC\STUDENT\HPC\cygwin\)
ssh -L 5901:zeus15:5901 username@zeus4.coventry.ac.uk
This will connect the port 5901 (i.e. VNC-server session :1 running on zeus15) to the port 5901 (you can hook it to any other if 5901 is already taken on your PC) of your local machine via login node zeus4. Obviously replace “username” by your HPC user name. Then, to connect to VNC session running on zeus15:1, just connect your vncviewer to localhost:1. If you need to forward any other non-privileged port from compute node to your local PC, just change the port numbers in the command above.
vncviewer localhost:1
PGI Community Edition includes a no-cost license to a recent release of the PGI Fortran, C and C++ compilers and tools for multicore CPUs and NVIDIA Tesla GPUs, including all OpenACC, OpenMP and CUDA Fortran features. The PGI Community Edition enables development of performance-portable HPC applications with uniform source code across the most widely used parallel processors and systems.
http://www.pgroup.com/products/community.htm
module load pgi/2017
Compilers:
pgc pgcc pgc++ pgf77 pgf90 pgf95 etc...
OpenMPI for PGI compilers
module load pgi/mpi/1.10.2/2017
OR
module load pgi/mpi/2.1.2/2017
Then mpi versions of PGI compilers will be available as
mpicc, mpic++, mpif90, mpif77...
Let’s compile parallel code with intel compiler, say version 2013
module load intel/13 (you can also choose intel/11/icc or intel/11/ifort)
module load intel/impi
mpicc -cc=icc main.c -o main.exe
mpif90 -fc=ifort main.f90 -o main.exe
module load intel/13
module load intel/impi
Launch on 8 CPUs:
mpirun -np 8 ./main.exe
Recent Comments