Anaconda Python distributions

Python versions

To use Python3 load the respective environmental module:

  • module load python/last

Then you can see the lists of the installed Python packages by issuing

conda list

If you need unlisted extra packages you can install them yourself in your home python folder. Refer to https://conda.io/docs/test-drive.html#managing-conda Sections 2,4.

or

https://conda.io/docs/user-guide/tasks/manage-environments.html

If you have trouble installing python packages, please contact me (Alex Pedcenko aa3025@coventry.ac.uk).

Submitting Python script to the SLURM queue

You can use a generic SLURM script for running a Python job (load appropriate python module first, i.e. “module load python/last” or “module load python/xxxxx” (check “module avail” to list them all)

The script below loads all necessary “modulefiles” (python and cuda), activates the needed for a particular job user-environment “myenvironment” and then launches the python file from the folder this job is being submitted. The “standard output” will be written to slurm-xxxx.out file in the same folder.

#!/bin/bash


#SBATCH --job-name="pyjob"

source ~/.bashrc
module load python/last
module load cuda/11.4
conda activate myenvirnment


python ./mypythonscript.py

save this script to your working folder (e.g. as python_job.slurm)

and mypythonscript.py can contain anything you want to run, e.g.

#!/usr/bin/env python

import tensorflow as tf
aaa=tf.config.list_physical_devices('GPU')
print(aaa)

and submit to SLURM queue with sbatch (e.g. we ask for 1 compute node, 1 task, 8 CPUs per task and 1GPU on centos8 node ):

sbatch -N1 -n1 -c 8 --constraint=centos8 --gres=gpu:1 python_job.slurm

Alex Pedcenko

css.php