Example batch job scripts for Puhti
Example job scripts for running different types of programs:
Note
If you use the scripts (please do!), do not forget to change the resources
(time, tasks etc.) to match your needs and to replace myprog <options>
with the executable (and options) of the program you wish to run as well
as <project> with the name of your project.
Serial
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=<project>
#SBATCH --partition=small
#SBATCH --time=02:00:00
#SBATCH --ntasks=1
#SBATCH --mem-per-cpu=4000
srun myprog <options>
OpenMP
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=<project>
#SBATCH --partition=small
#SBATCH --time=02:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=6
#SBATCH --mem-per-cpu=4000
# set the number of threads based on --cpus-per-task
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
srun myprog <options>
MPI
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=<project>
#SBATCH --partition=large
#SBATCH --time=02:00:00
#SBATCH --ntasks=80
#SBATCH --mem-per-cpu=4000
srun myprog <options>
Large MPI
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=<project>
#SBATCH --partition=large
#SBATCH --time=02:00:00
#SBATCH --nodes=8
#SBATCH --ntasks-per-node=40
#SBATCH --mem-per-cpu=4000
srun myprog <options>
MPI + OpenMP
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=<project>
#SBATCH --partition=large
#SBATCH --time=02:00:00
#SBATCH --ntasks=8
#SBATCH --cpus-per-task=10
#SBATCH --mem-per-cpu=4000
# set the number of threads based on --cpus-per-task
export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK
srun myprog <options>
Note
By default, running a single task per node with multiple threads using hpcx-mpi will bind all threads to a single
core and no speedup will be gained. This can be fixed by setting export OMP_PROC_BIND=true in your job script. This
will bind the threads to different cores. Another possibility is to turn off slurms core binding with the srun flag --cpu-bind=none.
Single GPU
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=<project>
#SBATCH --partition=gpu
#SBATCH --time=02:00:00
#SBATCH --ntasks=1
#SBATCH --cpus-per-task=10
#SBATCH --mem-per-cpu=8000
#SBATCH --gres=gpu:v100:1
srun myprog <options>
Multiple GPUs
#!/bin/bash
#SBATCH --job-name=example
#SBATCH --account=<project>
#SBATCH --partition=gpu
#SBATCH --time=02:00:00
#SBATCH --ntasks=4
#SBATCH --cpus-per-task=10
#SBATCH --mem-per-cpu=8000
#SBATCH --gres=gpu:v100:4
srun myprog <options>
Interactive with X11 graphics
Give this directly on the command line instead via a file and sbatch.
Note, as you may need to queue, it's convenient to ask for an email once the resources have been granted.
srun --ntasks=1 --time=00:10:00 --mem=1G --x11=first --pty \
--account=<project> --partition=small --mail-type=BEGIN \
myprog