Sections

 

Using Virtual Environments

Note: The recommended location for virtualenvs is in the /projectnb space of any project. The home directory is not recommended due to strict space and file number quotas.

Create virtualenv

Load the Python module version you wish to use and create a new virtual environment named mynewnev in your /projectnb space.

[rcs@scc1 ~] module load python3/3.10.12
[rcs@scc1 ~] virtualenv /projectnb/yourprojectname/venvs/mynewenv
New python executable in  /projectnb/yourprojectname/venvs/mynewenv/bin/python
Installing setuptools, pip, wheel...done.

 

Activate virtualenv

Activate mynewev. Note that once activated, the command prompt begins with (mynewenv) signifying you are inside the virtual environment.

[rcs@scc1 ~] source /projectnb/yourprojectname/venvs/mynewenv/bin/activate
(mynewenv) [rcs@scc1 ~]

 

Install into virtualenv

Install the Python package of interest into your virtualenv. In this example, we are installing the HTSeq into mynewenv.

(mynewenv) [rcs@scc1 ~] pip install htseq
Collecting htseq
Collecting numpy
Collecting pysam
Installing collected packages: numpy, pysam, htseq
Successfully installed htseq-0.12.4 numpy-1.19.1 pysam-0.16.0.1

 

Exporting and Distributing Virtual Environments

It may be useful to export an existing virtualenv to another workspace or share with others outside SCC Project (a.k.a. Unix group). If you would like to export or share the virtualenv with collaborators that cannot be given Unix group-read/execute permissions, follow the instructions below.

To be followed by the user exporting the virtualenv.

  1. Load a Python module for the version you wish to use. 
    [rcs@scc1 ~] module load python3/3.10.12
[rcs@scc1 ~] source /projectnb/yourprojectname/venvs/mynewenv/bin/activate
  2. Export list of packages via pip into a text document named requirements.txt. Some example packages in this environment are HTSeq, Numpy, and Pysam. 
    (mynewenv) [rcs@scc1 ~] pip freeze --local > requirements.txt
(mynewenv) [rcs@scc1 ~] cat requirements.txt
HTSeq==0.12.4
numpy==1.19.1
pysam==0.16.0.1
...
  3. Deactivate the current virtual environment. 
    (mynewenv) [rcs@scc1 ~] deactivate
[rcs@scc1 ~]

To be followed by the user re-creating the exported virtualenv.

  1. Build a new virtualenv called mynewenv_copy and activate it. 
    [rcs@scc1 ~] virtualenv </span /projectnb/yourprojectname/venvs/mynewenv_copy
New python executable in /projectnb/yourprojectname/venvs/mynewenv_copy/bin/python
Installing setuptools, pip, wheel...done.
[rcs@scc1 ~] source /projectnb/yourprojectname/venvs/mynewnev_copy/bin/activate
(mynewnev_copy) [rcs@scc1 ~]
  2. Install everything from the requirements.txt file. 
    (mynewnev_copy) [rcs@scc1 ~] pip install -r requirements.txt
Collecting htseq
Collecting numpy
Collecting pysam
Installing collected packages: numpy, pysam, htseq
Successfully installed htseq-0.12.4 numpy-1.19.1 pysam-0.16.0.1
  3. Test that the expected packages were installed and work in the new virtualenv. In this example, we are specifically testing the HTSeq package. 
    (mynewnev_copy) [rcs@scc1 ~] which htseq-count
/projectnb/yourprojectname/venvs/mysecondenv/bin/htseq-count
(mynewnev_copy) [rcs@scc1 ~] python
Python 3.10.12 (default, May 21 2023, 14:57:43) 
[GCC 4.8.5 20150623 (Red Hat 4.8.5-39)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>> import HTSeq

 

Using the –system-site-packages flag

When creating a virtualenv, you can add the --system-site-packages flag to have the existing Python packages in the SCC Python module also installed in your new virtualenv. This may be beneficial as this will automatically install many of the commonly used Python packages (NumPy, SciPy, etc.) into your virtualenv. If you would like to create an empty virtualenv, skip this option.

    1. Load the Python module for the version you wish to use. 
      [rcs@scc1 ~] module load python3/3.10.12
    2. Create a virtualenv named mynewenv_pack in your /projectnb space. 
      [rcs@scc1 ~] virtualenv --system-site-package /projectnb/yourprojectname/venvs/mynewenv_pack
New python executable in /projectnb/yourprojectname/venvs/mynewenv_pack/bin/python
Installing setuptools, pip, wheel...done.
    3. Add Jupyter and/or Spyder if you need them. These need to be reinstalled into the virtualenv if you are using the --system-site-package option for them to work as expected. 
      [rcs@scc1 ~] source mynewenv_pack/bin/activate
(mynewenv_pack)[rcs@scc1 ~] pip install --force jupyter notebook spyder

     

    Back to top

    Last updated: Loading…