|
| str | DEFAULT_PROJECT = 'hallb-pro' |
| | Default JLAB project/allocation.
|
| |
| str | DEFAULT_SITE = 'jlab/enp' |
| | Default swif2 site name, passed to 'swif2 create -site-name'.
|
| |
| str | DEFAULT_OS = 'el9' |
| | Default OS constraint, passed to add-job as '-constraint'.
|
| |
| | VALID_PARTITIONS = frozenset(['production', 'ifarm', 'priority', 'jupyter', 'gpu']) |
| | Valid slurm partitions on the JLAB farm (from 'sinfo').
|
| |
| dict | LEGACY_TRACK_PARTITIONS |
| | Legacy Auger "track" names (accepted by the old add-jsub) mapped to their closest valid slurm partition.
|
| |
|
| | _partition (self) |
| | Resolve the swif2 (slurm) partition.
|
| |
| | _run_swif2 (self, args, check=True, quiet=False) |
| | Run a single 'swif2' subcommand, echoing its (non-empty) output.
|
| |
| | _create_cmd (self) |
| |
| | _create_workflow (self) |
| | Ensure the swif2 workflow exists (add-job requires it to already exist).
|
| |
| | _add_job_cmd (self, job_params) |
| | Build the 'swif2 add-job' argument list for a single job.
|
| |
| | _job_inputs (self, job_params) |
| | Yield (local_dest, remote_uri) pairs for a job's inputs, applying swif2 URI schemes.
|
| |
| | _job_outputs (self, job_params) |
| | Yield (local_src, remote_uri) pairs for a job's outputs, resolving ptag sources and applying schemes.
|
| |
| | _write_job_script (self, job_id) |
| | Write an executable tcsh wrapper script that sets up the (c-shell) environment and runs the job, and return its absolute path.
|
| |
| | _create_job_xml (self) |
| |
| | _jsub (self, xml_filename) |
| |
| | _get_auger_ids (self, out) |
| |
| | _write_req (self, req, filename='temp.xml') |
| |
| | _create_req (self, req_name) |
| |
| | _create_job (self, params) |
| | Needed for resolving ptag output sources.
|
| |
| | _add_job (self, req, job_params) |
| |
| | _logfile (self, job_id) |
| | Get the base name of a log file for the job.
|
| |
| | _get_filtered_job_ids (self) |
| | Get a list of job IDs to submit based on parsed command line options and whether output files are being checked.
|
| |
| | _job_ids_missing_output (self, job_ids) |
| | Get a list of IDs for jobs that are missing output files.
|
| |
Submit jobs to the 'swif2' workflow system at JLAB.
The legacy 'swif2 add-jsub' command (which consumed an Auger XML request file) has been deprecated by JLAB and is being removed. This class now builds the workflow with 'swif2 create' and adds each job with 'swif2 add-job', mapping the parameters that the parent Auger class used to encode in XML directly onto add-job command-line flags, then starts the workflow with 'swif2 run'.
Note that 'swif2 create' fails if a workflow of the same name already exists. Existing workflows generated by this class should be fully cancelled and removed before resubmitting under the same name.
Definition at line 680 of file batch.py.
| _add_job_cmd |
( |
|
self, |
|
|
|
job_params |
|
) |
| |
|
protected |
Build the 'swif2 add-job' argument list for a single job.
This reproduces, as add-job flags, the per-job information the parent Auger class used to write into the request XML: resource limits, inputs, outputs, stdout/stderr, the job name, and the command to run.
- Parameters
-
| job_params | raw job data dict from the job store |
- Returns
- list of arguments following the 'swif2' executable
Definition at line 852 of file batch.py.
Ensure the swif2 workflow exists (add-job requires it to already exist).
If a workflow of the same name already exists this is not an error: the create is skipped and jobs are added to the existing workflow. Pass –recreate to cancel and recreate it instead. Any other create failure is fatal. The create output is captured and only echoed on success or a genuine failure, so the benign 'already exists' case does not surface swif's scary error text.
Definition at line 827 of file batch.py.
| _job_inputs |
( |
|
self, |
|
|
|
job_params |
|
) |
| |
|
protected |
Yield (local_dest, remote_uri) pairs for a job's inputs, applying swif2 URI schemes.
http inputs are skipped (they are not staged by swif), matching the previous Auger behavior.
Definition at line 906 of file batch.py.
Resolve the swif2 (slurm) partition.
The old Auger "track" (debug mode -> 'debug', else the queue, else the 'production' default) is validated against the farm's real partitions: legacy track names are remapped to a valid partition with a warning, and anything else unknown is a hard error.
- Returns
- a valid slurm partition name
Definition at line 743 of file batch.py.
Convert a filesystem path to a swif2 remote URI.
Tape paths get the 'mss:' scheme; ordinary filesystem paths (e.g. /lustre, /work) are left bare, matching both the old Auger behavior and the remote URIs observed on live JLAB workflows (swif canonicalizes tape URIs to 'mss:jlab:...' on its own). Paths that already carry a scheme are returned unchanged.
Definition at line 933 of file batch.py.
| _write_job_script |
( |
|
self, |
|
|
|
job_id |
|
) |
| |
|
protected |
Write an executable tcsh wrapper script that sets up the (c-shell) environment and runs the job, and return its absolute path.
The script contents reproduce the old Auger <Command> CDATA block. It is written to the log dir (on /farm_out, which is shared with the compute nodes) and passed to add-job as the job command; passing a script path avoids the shell metacharacters that swif truncates out of an inline command string.
- Parameters
-
- Returns
- absolute path to the generated script
Definition at line 946 of file batch.py.
Public Member Functions inherited from Auger
1.9.8