The module package is available on the Shared Computing Cluster, allowing users to access non-standard tools or alternate versions of standard packages. This is also an alternative way to configure your environment as required by certain packages. Specific modules can be loaded and unloaded as required. The module command is provided by the Lmod software, developed at the Texas Advanced Computing Center.

The version of Python that is available without loading a module is an older version (2.7.5) that is intended for system usage only. Load a module in order to use Python.

Table of Contents

Common Commands

module list List currently loaded modules.
module avail List available packages.
module spider List available packages in a different format.
module help [modulefile] Description of specified module.
module show [modulefile] Displays information about specified module, including environment changes, dependencies, software version and path.
module load [modulefile] Loads module or specifies which dependencies have not been loaded.
module unload [modulefile] Unloads specified module from environment.
module purge Unloads all loaded modules

More information can be found in the Module man page.

Using Common Commands

module list

scc4% module list
No Modulefiles Currently Loaded.

module avail

List all available packages.

scc4% module avail

--------------------------------- /share/module.7/bioinformatics --------------------------------- 
   2brad_denovo/2019-01-22_giteec5016            meshclust2/2.1.0
   2brad_gatk/2019-01-22_git1fcc9e8              metal/2010-02-08
   admixture/1.3.0                               metal/2011-03-25                (D)
   angsd/0.923                                   metalge/2010-02-08
   annovar/2018apr                               minimac2/2014-09-15
   artemis/18.0.3                                minimac3/2.0.1
   augustus/3.3.2                                minimac4/1.0.0
   bamtools/2.5.1                                mirdeep2/0.1.0
   bamutil/1.0.14                                mixcr/3.0.3
   basemount/0.15.103.3011                       mmap/2018-04-07
   basespace-cli/0.8.12.590                      morgan/3.2
   basespace-cli/0.9.17                          morgan/3.4                      (D)
   basespace-cli/0.10.8                   (D)    mosdepth/0.2.6
   bayescan/2.1                                  mothur/1.35.0
   bbmap/38.16                                   multiqc/1.6
   bcbio/1.1.1                                   mummer/3.23

You can also search for packages by name. For example to see all versions of ‘tensorflow’, you can search for the word ‘tensorflow’ or a substring like ‘tensor’,

scc4% module avail tensor

--------------------------------- /share/module.7/machine-learning ------------------------------------------ 
   tensorflow/1.12    tensorflow/1.13.1    tensorflow/1.15.0 (D)    tensorflow/2.0.0

  Where:
   D:  Default Module

module spider

List available packages in a different format.

scc4% module spider

-----------------------------------------------------------------------------------------------------------
The following is a list of the modules currently available:
-----------------------------------------------------------------------------------------------------------
  2brad_denovo: 2brad_denovo/2019-01-22_giteec5016

  2brad_gatk: 2brad_gatk/2019-01-22_git1fcc9e8

  3d-ice: 3d-ice/2.2.6

  R: R/3.0.0, R/3.5.1, R/3.6.0_intel-2018, R/3.6.0, R/3.6.1_intel-2019

  admixture: admixture/1.3.0

  afni: afni/17.0.18-omp, afni/19.1.00-omp, afni/19.1.00, afni/2011-12-21.1014-xorg, afni/2011-12-21.1014

  amber: amber/16

  amgx: amgx/2019-12-13_gitb3101ff
    AmgX is a GPU accelerated core solver library that speeds up computationally intense linear solver
    portion of simulations.

  anaconda2: anaconda2/5.2.0

  anaconda3: anaconda3/5.2.0

You can also search for packages by name. For example to see all versions of ‘tensorflow’, you can search for the word ‘tensorflow’ or a substring like ‘tensor’,

scc4% module spider tensor

-----------------------------------------------------------------------------------------------------------
  tensorflow:
-----------------------------------------------------------------------------------------------------------
     Versions:
        tensorflow/1.12
        tensorflow/1.13.1
        tensorflow/1.15.0
        tensorflow/2.0.0

-----------------------------------------------------------------------------------------------------------
  For detailed information about a specific "tensorflow" module (including how to load the modules) use the module's full name.
  For example:

     $ module spider tensorflow/2.0.0
-----------------------------------------------------------------------------------------------------------

module help [modulefile]

scc4% module help gcc
------------------------------------ Module Specific Help for "gcc/8.3.0" -------------------------------------

gcc 8.3.0: GNU Compiler Collection (GCC)

C, C++, and Fortran compilers.

For more information on gcc, please see https://gcc.gnu.org/

The following environment variables are provided for convenience:
$SCC_GCC_DIR -- Base Package Directory
$SCC_GCC_BIN -- Package Executables Directory
$SCC_GCC_LIB -- Package Libraries Directory
$SCC_GCC_LICENSE -- Package License Information

module show [modulefile]

scc4% module show gcc
-----------------------------------------------------------------------------------------------------------
   /share/module.7/programming/gcc/8.3.0.lua:
-----------------------------------------------------------------------------------------------------------
help([[
gcc 8.3.0: GNU Compiler Collection (GCC)

C, C++, and Fortran compilers.

For more information on gcc, please see https://gcc.gnu.org/

The following environment variables are provided for convenience:
$SCC_GCC_DIR -- Base Package Directory
$SCC_GCC_BIN -- Package Executables Directory
$SCC_GCC_LIB -- Package Libraries Directory
$SCC_GCC_LICENSE -- Package License Information

]])
whatis("Name:         gcc")
whatis("Version:      8.3.0")
whatis("Description:  GNU Compiler Collection (GCC)")
whatis("URL:          https://gcc.gnu.org/")
whatis("Categories:   programming")
whatis("Keywords:     centos7, programming, gnu, compiler, c, c++, fortran")
setenv("SCC_GCC_DIR","/share/pkg.7/gcc/8.3.0/install")
setenv("SCC_GCC_BIN","/share/pkg.7/gcc/8.3.0/install/bin")
setenv("SCC_GCC_LIB","/share/pkg.7/gcc/8.3.0/install/lib64")
setenv("SCC_GCC_LIB32","/share/pkg.7/gcc/8.3.0/install/lib")
setenv("SCC_GCC_LICENSE","/share/pkg.7/gcc/8.3.0/install/COPYING")
prepend_path("PATH","/share/pkg.7/gcc/8.3.0/install/bin")
prepend_path("LD_LIBRARY_PATH","/share/pkg.7/gcc/8.3.0/install/lib")
prepend_path("LD_LIBRARY_PATH","/share/pkg.7/gcc/8.3.0/install/lib64")

Basic Command Sequence

Below is a basic sequence of module commands to use to get started using modules on the SCC.

  1. List all available modules by executing “module avail” command. The results are a list of module names that can be loaded. 
    [username@scc1 ~]$ module avail
---------------------------- /share/etc/modulefiles ----------------------------
2brad_denovo/2019-01-22_giteec5016            forestpmplot/1.0.1                        plink/1.07                        (D)
2brad_gatk/2019-01-22_git1fcc9e8              gatk/3.8-1                                plink/1.90b6.4
admixture/1.3.0                               gatk/4.0.11.0                             plink/2.00a1LM
angsd/0.923                                   gatk/4.0.12.0                             plink/2.00a2.3
annovar/2018apr                               gatk/4.1.2.0                       (D)    plink/2.0
annovar/2019oct24                      (D)    gatk/4.1.3.0                              plinkseq/0.10
anvio/6.1                                     gatk/4.1.4.1                              postgap/2020-05-11-git0453c91
...                                   ...
  2. Filter the result by adding keywords to “module avail” command, such as “julia“. 
    [username@scc1 ~]$ module avail julia
------------------------- /share/module.7/programming --------------------------
   julia/0.7.0    julia/1.0.2    julia/1.3.0    julia/1.4.2    julia/1.5.0 (D)

  Where:
   D:  Default Module

Use "module spider" to find all possible modules.
Use "module keyword key1 key2 ..." to search for all possible modules matching
any of the "keys".
  3. Load the module using “module load” and append the module name. The following example shows loading Julia version 1.5.0. 
    [username@scc1 ~]$ module load julia/1.5.0
  4. Confirm the module is loaded by executing “module list“. 
    [username@scc1 ~]$ module list
    
Currently Loaded Modules:
  1) julia/1.5.0
  5. Now you can run your application. In this example we will check the version of Julia loaded. 
    [username@scc1 ~]$ julia -version
julia version 1.5.0
  6. When you are done with the application, unload the module using “module unload” command with the name of the module you want to remove. 
    [username@scc1 ~]$ module unload julia/1.5.0
  7. Confirm the module was unloaded by running “module list” command. 
    [username@scc1 ~]$ module list
No modules loaded

Scripts and Batch Commands

To ensure that modules are loaded correctly within a script file add -l option to the first line of your script, i.e.:

#!/bin/bash -l

When including modules in scripts and batch submission commands (via qsub), it is the best practice to specify modules and applications by version number. Over time, newer versions of the applications will be installed. Specifying versions of applications ensures that in the future jobs will run the same way as they do now.

Example:

module load tophat/tophat-2.0.4_gnu446

rather than

module load tophat

Modules Best Practices

Below are best practices for using modules

  • RCS is always adding new and updated software modules to the SCC, but newer versions are not always backwards compatible and can break existing scripts or workflows. When using the “module load” command, one should always use the full module name (including version number) to ensure the same modules are loaded each time you work on your analysis.
  • In order to reduce the chance of unexpected behavior caused by module conflicts, only load modules that are needed for the current script or workflow you are running.
  • Avoid including “module load” commands in your .bashrc or .bash_profile files for convenience, as we have found this causes instability and other issues for our researchers. As an alternative, create a bash script with the module load commands and source it each time, to load the modules needed.

How to redirect module output

This answer also addresses: How do I grep ‘module avail’? The short answer is: redirect standard error (stderr) to standard output (stdout). If you are using the bash shell, this is done using the ‘2>&1‘ syntax. For example, to search for python in the available modules, you can type:

scc4% module -t avail 2>&1 | grep -i python

In [t]csh, the command is a bit more complicated, because there is no trivial way to redirect stderr. The following example calls the module command in a separate process and then it pipes the stderr and stdout to grep:

scc4% (module -t avail) |& grep -i python
Note: module avail already has built-in search capabilities which avoids most of the need to use tools like grep.