The Intel Distribution for Python behaves like regular Python, but leverages Intel technologies to speed up many of the core python libraries, including NumPy, SciPy, Pandas, Scikit-Learn, Jupyter, matplotlib, and mpi4py. This distribution also integrates Intel Math Kernel Library (Intel MKL), Intel Data Analytics Acceleration Library (DAAL) and pyDAAL, Intel MPI Library, and Intel Threading Building Blocks (TBB). The following modules offer significant speedups for some computational workloads at the cost of potential incompatibility with other python packages.
Sections
Using Python3-Intel
We recommend creating a virtualenv with one of the Python3-Intel modules. It’s also recommended to use the --system-site-packages flag to automatically install the Python packages we’ve already installed in the module.
[rcs@scc1 ~] module load python3-intel/2022.2.0
[rcs@scc1 ~] virtualenv --system-site-packages /projectnb/yourprojectname/venvs/mynewenv_intel
New python executable in /projectnb/yourprojectname/venvs/mynewenv_intel/bin/python
Installing setuptools, pip, wheel...done.
[rcs@scc1 ~] source /projectnb/yourprojectname/venvs/mynewenv_intel/bin/python
(mynewenv_intel) [rcs@scc1 ~]
Optimizing on the SCC
The optimizations available in the Intel Distribution of Python depend both OpenMP and Intel multi-processing libraries. Two environment variables are used coordinate the automatic parallel processing: OMP_NUM_THREADS and MKL_NUM_THREADS. It is important to understand the impact of these variables on your code and define appropriate values when running your code. Most commonly, you should set the value equal to the number of slots ($NSLOTS) your job requests, but not always. Please read Intel’s guidance on threaded applications.
| Module Name | Description | Default Value | Recommended |
|---|---|---|---|
OMP_NUM_THREADS |
Sets the number of threads for OpenMP | 1 | $NSLOTS |
MKL_NUM_THREADS |
Sets the number of threads for Intel Math Kernel Library | 1 | $NSLOTS |
OPENBLAS_NUM_THREADS |
Sets the number of threads for OpenBLAS | 1 | $NSLOTS |
NUMBA_NUM_THREADS |
Sets the number of threads for NUMBA | 1 | $NSLOTS |