alliance-slurm

Alliance (DRAC) Slurm

Alliance clusters run Slurm with a custom Lua submission layer that auto-routes jobs and a prolog that injects environment variables. Hardware varies by cluster — always discover before assuming. Auth is managed via CCDB; software modules come from CVMFS.

For cluster policies, storage quotas, and account questions, check alliance-docs first.

Discovery — never hardcode, always list

Run these first on any new cluster. Never guess account names, GPU types, partition names, or QOS values.

# Partitions and time limits
sinfo -s                                            # TIMELIMIT column is authoritative

# GPU GRES inventory (look for composite entries, e.g. gpu:l40s:4,shard:l40s:16)
sinfo -h -o "%G" --Node | sort -u
sinfo -h -o "%N %G %f" --Node | sort -u            # adds features (softmig, etc.)

# Full GRES detail on a specific node
scontrol show node <node> | grep -E 'Gres=|CfgTRES='

# Accounts available to you
sacctmgr show assoc user=$USER format=Account,DefaultAccount,QOS -p

# QOS limits (MaxWall / MaxTRES are often blank; prefer sinfo -s for walltime)
sacctmgr show qos format=Name,MaxWall,MaxTRES,Priority -p

# Fairshare / software
sshare -U
module avail && module spider <name>

Always run GPU discovery before writing --gres flags. GPU type names (v100l, a100, l40s, h100, …) differ by cluster and node.

Partitions — usually omit

A Lua plugin auto-routes jobs based on --time. Omit --partition unless you have a concrete reason (e.g. forcing CPU-only). Setting it with a mismatched --time causes instant rejection.

• Time tiers go up to a 7-day maximum; requests over 7 days are rejected.
• srun/salloc always route to the interactive partition. Check actual limits with sinfo -s — GPU interactive is often longer than CPU interactive (e.g. 8 h vs 3 h).

Key flags

Flag	Notes
--account=<acct>	Required when you have multiple accounts or must charge a specific project. Check with sacctmgr show assoc user=$USER; omit only if your default account is correct.
--time=HH:MM:SS	Default is 60 min. Drives partition routing. Always set.
--mem=...	Default is very low, cgroup-enforced. Always set.
--gres=gpu:<type>:N	Use exact type string from sinfo discovery. Bare gpu:N is unreliable and is rejected on general-purpose clusters (fir, nibi, rorqual, and similar). Always use the full type string.
--cpus-per-task=N	Default is 1.