cProfile: Python profiler
cProfile is a built-in profiler for Python programs. There are two ways to use the profiler. Within the code (or from the interpreter):
import cProfile
cProfile.run('functba(list_parameters)')
Now the script can be ran as a normal Python job. This will give information about how long and how many times the function gets called.
Alternatively cProfile can also be invoked as a script to profile another script:
python -m cProfile [-o output_file] [-s sort_order] myscript.py
The results can be printed out or saved to a file. By default they are ordered by name, but other options are available, too.
A report saved in a file can e.g. be visualized and interpreted by a
graphical tool such as pyprof2calltree. Here is an example of a profiling
done for a function:
93 function calls in 0.065 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.065 0.065 <string>:1(<module>)
1 0.000 0.000 0.028 0.028 _numpy_fft.py:1086(irfftn)
1 0.000 0.000 0.003 0.003 arraypad.py:102(_do_append)
2 0.000 0.000 0.000 0.000 arraypad.py:107(_prepend_const)
2 0.000 0.000 0.004 0.002 arraypad.py:136(_append_const)
1 0.000 0.000 0.008 0.008 arraypad.py:964(pad)
1 0.000 0.000 0.065 0.065 compute_correlations.py:4(compute_correlations)
1 0.032 0.032 0.065 0.065 normxcorr2.py:33(normxcorr2)
1 0.000 0.000 0.065 0.065 {built-in method builtins.exec}
1 0.003 0.003 0.003 0.003 {built-in method numpy.concatenate}
2 0.002 0.001 0.002 0.001 {built-in method numpy.copyto}
1 0.002 0.002 0.002 0.002 {built-in method numpy.where}
1 0.000 0.000 0.000 0.000 {built-in method numpy.zeros}
2 0.000 0.000 0.000 0.000 {method 'astype' of 'numpy.ndarray' objects}
2 0.003 0.002 0.003 0.002 {method 'copy' of 'numpy.ndarray' objects}
1 0.020 0.020 0.028 0.028 {mkl_fft._pydfti.irfftn_numpy}
Keep in mind that running the profiler has to be done in the same way as running a (puhti or mahti) batch or interactive job.