Content Comparison

...

Memory request scenarios

• If you have run the program before on SCU resources you can view the job's memory statistics. From there, you can view the job's Maximum used memory and request a small amount over it for your next iteration of the job. See: Requesting appropriate amounts of resources
• If you have never run the program before; refer to the program's documentation for recommended memory requirements. If none exist, start with 8GB and increase if necessary.

...

Slurm allocates resources based on the listed hardware requirements in your srun command or sbatch script. It is imperative that users neither exaggerate nor underestimate their resource requirements. Slurm provides tools and resources to help users understand their jobs' resource requirements. See Consequences of requesting inappropriate resources.

Memory 

RAM is the SCU's most consumed resource. Therefor, the SCU recommends that your jobs request memory no more than 4GB or 20% greater than your previous jobs' maximum used memory (whichever is higher). This command can be used to query your completed jobs' memory allocation vs their maximum memory use in the past 7 days (modify date field as desired): 

...

Consequences of requesting inappropriate resources 

Slurm allocates and isolates your requested resources. Therefor, if resources are either overestimated, or underestimated, it could detriment the timeliness of your workflows and others'.

1. Unused resources are resources that could have been used to run other jobs, including yours. If your job requests 100GB of RAM but only uses 12GB, the unused 88GB will have been made unavailable to all other jobs in the queue; slowing down the cluster's churn rate, and causing your jobs and others' to lag behind.
2. Unused resources are counted towards your FairShare allocation, lowering your priority for your future jobs. See FairShare. 
3. Underestimated resources may cause slurm to kill your jobs. So it is important to neither underestimate nor greatly overestimate the resources needed. Slurm will only kill jobs that exceed their memory requests. Threads are individually allocated and isolated so your jobs cannot exceed their allocated thread count.

...

When utilizing MPI, resource requests in Slurm should be allocated differently. Slurm works well with MPI and other parallel environments and as such, srun can be called directly to run mpi programs (see sample script below).

Note: Code and programs must be compiled with explicit support for MPI to utilize MPI capabilities. Also, this does not apply to relion MPI calculations. For SCU documentation on Relion, see: Relion

...

For CPU and Memory statistics, see Requesting appropriate amounts of resources

...

FairShare

...

1. Account: Name of lab in Slurm's database.
2. User: When using sshare -a, slurm displays the fairshare calculation for all users in all labs. This can be used for more thorough accounting.
3. RawShares: The amount of shares a lab has been allotted. This is reached by calculating total athena storage * 100 = RawShares
4. NormShares: This is the percentage of a lab's RawShares to the total shares for the entire cluster. This number is used by slurm to allocate n percentage of cluster usage to said lab.
5. RawUsage: The amount of compute usage a lab has run or has requested in their jobs. sshare -l  will display a column which lists exact CPU and RAM usages.
6. EffectvUsage: The percentage of actual cluster use by the lab. This is compared with the lab's NormShares to determine the lab's FairShare score in the next column. If EffectvUsage is greater than the lab's NormShares, the fairshare will always be < 0.5. The reverse is also true. See FairShare Score
7. FairShare: The lab's resulting FairShare score. See FairShare Score.

Halflife

Slurm FairShare calculations have a halflife of 7 days. So if a lab exhausts its allocated usage and its resulting FairShare is = 0, it's FairShare should reset to 0.5 if 7 days pass with no usage.

...