The module academic-ml provides a set of conda environments with several different machine learning (ML) libraries installed, in addition to a large amount of useful Python software. It was created to facilitate the use of Python ML libraries on the SCC for academic classes. The module can, of course, be used for purposes beyond academic classes.

A new version of the academic-ml module will be installed before the start of the Spring and Fall semesters so that the latest updates to the various libraries will be available.

Sections

• Using the academic-ml Module
• Customizing the academic-ml Module Environments
• Intel Extensions for PyTorch

 

Using the academic-ml Module

This module includes two conda environments based on the GPU-enabled PyTorch and Tensorflow libraries. The GPU capabilities are automatically used when these environments are used on GPU-equipped compute nodes :

• spring-2025-pyt  – PyTorch 2.5.1
• spring-2025-tf  – Tensorflow 2.18.0
• spring-2025-jax  – Jax 0.4.38

Aside from the core ML software and associated libraries, these environments are identical, and a summary of the available software is provided. Each one provides development software such as Jupyter Notebooks, Spyder, the Nvidia CUDA SDK,  ML libraries like Transformers, and common Python libraries such as Numpy, Scipy, and Pandas. A complete list of every installed library with all versions is available by running the conda list command after activating the environments.

To use any of these environments, load the miniconda module, then the academic-ml module. First-time users of the miniconda module will prompted to run a setup script. See the miniconda documentation for more details:

module load miniconda
module load academic-ml/spring-2025
# see a summary of installed libraries:
module help academic-ml/spring-2025

Loading the academic-ml module in a terminal prints a message with instructions to activate these environments, and these commands are shown here:

# Activate the PyTorch environment:
conda activate spring-2025-pyt
# Or activate the Tensorflow environment:
conda activate spring-2025-tf
#Or activate the Jax-based environment run:
conda activate spring-2025-jax

When requesting a Jupyter Notebook, Jupyter Lab, or Spyder OnDemand session, enter the conda activation command in the Pre-Launch Command field. This screenshot shows an example of starting a Jupyter Notebook session with the spring-2025-tf environment.

Customizing the academic-ml Module Environments

If you want to customize the conda environments, for example to add additional libraries for an academic course or for your own work, you will need to re-create one of these conda environment in your project’s /projectnb (or /restricted/projectnb) directory. Once the environment is recreated you can modify it as you like. There are two environment variables set when the module is loaded that provide the location of a .yml file for each of the two environments:

• $SCC_ACADEMIC_ML_YML_PYT – PyTorch .yml file
• $SCC_ACADEMIC_ML_YML_TF – Tensorflow .yml file
• $SCC_ACADEMIC_ML_YML_JAX – Jax.yml file

These are used to create your project’s copy of the environment.  As an example, let’s make a clone of one of the environments and add an additional library to it for use in a course.

module load miniconda
# Using the default (i.e. latest) miniconda module 
# lets you use the mamba command.
# mamba is a much faster replacement for the conda command.
module load academic-ml/spring-2025
# make sure you are in the class /projectnb directory. 
cd /projectnb/course_number

# Change to the materials subdirectory
cd materials
# make a software subdirectory
mkdir software
cd software

# Make a clone of the spring-2025-pyt PyTorch-based environment. 
# This will use ~17GB of space.
mamba env create -p ./class-custom-pyt -f  $SCC_ACADEMIC_ML_YML_PYT 
# Wait 30-45 minutes...
conda activate  ./class-custom-pyt

# Install the emoji library, for adding emojis to your Python print statements.
mamba install emoji -c conda-forge

# ***FOR TENSORFLOW ENVIRONMENTS ONLY***
# Run this extra command which configured your environment to set library
# paths correctly. The argument is the full path to your new environment.
$SCC_ACADEMIC_ML_DIR/scripts/fix_tf_envs.sh  /projectnb/course_number/materials/software/class-custom-tf
 
# In the future, to use your custom conda environment, use:
conda activate /projectnb/course_number/materials/software/class-custom-pyt  

Intel Extensions for PyTorch

The spring-2025-pyt environment includes the Intel Extensions for PyTorch library. This may improve the performance of PyTorch when running on CPU-only compute nodes, as might be done while developing or debugging codes, or for doing inference calculations. Intel provides extensive documentation on how to apply this library.

Here is a short example of running inference on a pre-trained model. The Intel Extensions for  PyTorch are automatically loaded and applied if there is no GPU available:

import torch
import torchvision.models as models

model = models.resnet50(weights='ResNet50_Weights.DEFAULT')
model.eval()
data = torch.rand(1, 3, 224, 224)

if not torch.cuda.is_available():
  # This will only load if a GPU is not available.
  # CPU-only, optimize for CPUs.
  import intel_extension_for_pytorch as ipex
  import os
  # Apply Intel optimizations
  model = ipex.optimize(model)
  # enable PyTorch multi-threading based on the number
  # of cores assigned to this job on the SCC
  torch.set_num_threads(int(os.environ.get('NSLOTS',1)))

with torch.no_grad():
  model(data)

print("Execution finished")

Back to top