File System Structure

Filesystem Structure

The Linux file system starts at the “root” (/) and extends forward into directories that can contain both subdirectories and files. A series of descending directories can be strung together separated by forward slash characters “/” to indicate a location on the filesystem. This string of folders is called a “path” and will look like /project/projectname/subfolder/file.  This path structure is much like those of other common operating systems. Below are the most common file storage locations you will use on the SCC.

Places to Store Your Files

Users on the SCC are automatically granted several locations to store their files. Our overall file storage system is described here. Most users will be primarily storing files in three areas, all of which are generally accessible from all of the login and compute nodes; the exception is that the /restricted/ partitions are only accessible from the scc4.bu.edu login node and all of the compute nodes. We have guidelines for what data should be stored in each partition.

• Home Directory – This directory is entirely controlled by you and the default permissions are that nobody else can see or otherwise access your files. Home directories have a quota of 10 GB and this will generally not be increased. You will naturally store files directly related to your account here, such as dotfiles. It is also commonly used to store personal files. Although it is possible to do work in your home directory if it fits within the 10GB limit, we recommend you use Project Disk Space in case you end up needing more space than you anticipate. Home directories are both protected by Snapshots and backed-up off-site.
• Backed-Up Project Disk Space – Projects are by default granted 50 GB of space under /project/project_name/ (or /restricted/project/project_name/ for most BUMC projects). This number can be increased to a maximum of 200 GB at the request of the project leader(s), but it can not go beyond that. This data is both protected by Snapshots and backed-up off-site. Depending on the workflow of the project, a reasonable approach is to keep code and files you hand-edit in /project/ and files downloaded or generated by code or applications in /projectnb/.
• Not Backed-Up Project Disk Space – Projects are by default granted 50 GB of space under /projectnb/project_name/ (or /restricted/projectnb/project_name/ for most BUMC projects). This can be increased for free to a maximum allocation of 1000 GB and additional Not Backed-Up space can be purchased through either Buy-In or Storage-as-a-Service. Despite the name for this space, it is protected by both hardware RAID (protecting against disk failures) and daily Snapshots (protecting against accidental deletion of files). You will want to use this space for any large quantities of data you have.
• You can see which projects you belong to by running the command groups and see how much space each project has available and where by running pquota. Note that there are a few special groups like gaussian that do not have any disk space associated with them; pquota will tell you where you have disk space you can use. However, for those with space in /restricted/project/ or /restricted/projectnb/ groups will list the directories as /rproject/project_name/ and /rproject/project_name/ but you must type the full /restricted/... to access them.

For those who need more than 200 GB of backed up space, there is also the option to use Second Tier Ancillary Storage Heap (STASH) storage space but you will need to maintain your own back-ups of the data to this space.

Temporary Storage using Scratch Disk Space

All SCC nodes have a fairly large local /scratch directory that users can use for temporary storage of files. More details on this Scratch Space are available here.