sacctmgr
Manage Slurm account and cluster information
TLDR
Show current configuration
Add a cluster to the slurm database
Add an account to the slurm database
Show details of user/association/cluster/account using a specific format
SYNOPSIS
sacctmgr [GLOBAL_OPTIONS] COMMAND [COMMAND_OPTIONS] [ARGUMENTS]
Common COMMANDs:
add OBJECT [PARAMETERS]
modify OBJECT [PARAMETERS]
delete OBJECT [PARAMETERS]
show OBJECT [PARAMETERS]
list OBJECT [PARAMETERS]
Common OBJECTs:
account | user | association | cluster | qos | coordinators
Example Usage:
sacctmgr add account new_department Description="Account for R&D"
sacctmgr modify user jdoe DefaultAccount=new_department
sacctmgr show association user=jdoe format=account,user,cluster,partition
PARAMETERS
--clusters=<string>
Specifies the clusters to operate on. Multiple cluster names can be comma-separated.
--federation
Operates on the entire federation, if Slurm federation is configured.
--immediate
Commits changes to the database immediately. This is often the default behavior for 'add', 'modify', and 'delete' commands.
--parsable
Outputs results in a machine-readable format with fields separated by a vertical bar ('|'). Useful for scripting.
--parsable2
Outputs results in an enhanced machine-readable format, similar to --parsable but with more details and often better suited for complex parsing.
--quiet
Suppresses informational messages, showing only errors or requested output.
--readonly
Prevents any modifications to the Slurm database, useful for testing commands or viewing configurations safely.
--help
Displays a brief help message for the sacctmgr command or a specific subcommand.
--version
Displays the version of sacctmgr being used.
--name=<string>
Used with show commands to filter results by name, e.g., an account name or user name.
--level=<int>
Specifies the depth level for 'show tree' commands, controlling how many levels of hierarchy are displayed.
--noheader
Suppresses the header line in the output of 'show' commands, useful for scripting.
--port=<port>
Specifies the port number to connect to the slurmdbd daemon.
--local
Ensures that sacctmgr connects only to the local slurmdbd daemon, avoiding any potential remote connections configured.
DESCRIPTION
sacctmgr is a powerful administrative command-line utility for managing the Slurm Workload Manager's accounting database. It enables system administrators to define and control resource allocation policies by manipulating accounts, users, associations (linking users to accounts on specific clusters), clusters, and Quality of Service (QOS) definitions. This tool is essential for setting up fairshare policies, enforcing usage limits, and organizing the hierarchical structure of resource access within a Slurm cluster or federation of clusters.
Through sacctmgr, administrators can add new accounts, modify existing user permissions and limits, delete obsolete entries, and review the current configuration of the Slurm database. It interacts directly with the Slurm Database Daemon (slurmdbd), ensuring that all changes are immediately reflected in the system's resource management logic. Its comprehensive set of commands and options provides granular control over who can access what resources and under what conditions, making it a cornerstone for effective Slurm cluster management.
CAVEATS
Using sacctmgr requires administrative privileges, typically root or the SlurmUser. Because it directly modifies the Slurm accounting database, extreme caution must be exercised; incorrect commands can disrupt resource allocation, job scheduling, or user access across the cluster. It's crucial to understand Slurm's account hierarchy, QOS definitions, and fairshare principles before making significant changes. Always verify your commands, especially when performing 'delete' operations. While changes are generally immediate, it's wise to double-check their effect using 'show' commands.
UNDERSTANDING SLURM ACCOUNTING HIERARCHY
Slurm uses a hierarchical accounting system to organize users and resource consumption. At the top are accounts, which can represent departments, projects, or groups. Accounts can be nested, forming a tree structure (e.g., 'department_A' -> 'project_X'). Users are associated with one or more accounts, and their resource usage is charged against these associations. This hierarchy, managed largely through sacctmgr, is fundamental to implementing fairshare scheduling and managing resource limits effectively across an organization.
INTEGRATION WITH QUALITY OF SERVICE (QOS)
Quality of Service (QOS) objects, managed by sacctmgr, define various job submission and execution parameters such as maximum job run time, maximum number of jobs, priority, and preemption rules. QOS can be assigned to accounts, users, or associations, providing a flexible mechanism to enforce resource policies. Administrators use sacctmgr to create new QOS definitions, modify existing ones, and assign them to specific entities, profoundly influencing how jobs from different groups or users are prioritized and constrained.
HISTORY
sacctmgr is an integral part of the Slurm Workload Manager, a highly scalable and fault-tolerant cluster management and job scheduling system for large and small Linux clusters. Slurm was originally developed by Lawrence Livermore National Laboratory (LLNL) and released as open-source in 2002. As Slurm evolved to manage complex resource allocation and accounting, the need for a dedicated administrative tool for its database became apparent. sacctmgr was introduced to provide a robust, command-line interface for managing the hierarchical accounting structures, users, clusters, and Quality of Service (QOS) policies stored within the slurmdbd database. Its development has paralleled Slurm's growth, incorporating features like federation support and more granular control over resource limits, making it a cornerstone for Slurm administrators.