Generic batch processing interface.
More...
|
| def | __init__ (self) |
| |
| def | parse_args (self, args) |
| | Parse command line arguments and perform setup. More...
|
| |
| def | submit_job (self, job_id) |
| | Submit a single batch job and return the batch ID. More...
|
| |
| def | submit (self) |
| | This is the generic batch submission function which gets a list of jobs to run based on command line arguments and submits them individually. More...
|
| |
| def | default_rundir (self, job_id=None) |
| |
| def | build_cmd (self, job_id) |
| | This is the basic implementation of building a command to run the job from a batch system. More...
|
| |
Generic batch processing interface.
Definition at line 35 of file batch.py.
◆ __init__()
◆ _get_filtered_job_ids()
| def _get_filtered_job_ids |
( |
|
self | ) |
|
|
private |
Get a list of job IDs to submit based on parsed command line options and whether output files are being checked.
Definition at line 209 of file batch.py.
◆ _job_ids_missing_output()
| def _job_ids_missing_output |
( |
|
self, |
|
|
|
job_ids |
|
) |
| |
|
private |
Get a list of IDs for jobs that are missing output files.
Definition at line 226 of file batch.py.
◆ _logfile()
| def _logfile |
( |
|
self, |
|
|
|
job_id |
|
) |
| |
|
private |
Get the base name of a log file for the job.
Definition at line 190 of file batch.py.
◆ _outputs_exist()
| def _outputs_exist |
( |
|
job | ) |
|
|
staticprivate |
Check if all output files exist for the given job.
This is not the job ID but the full JSON job data.
Return False when first missing output is found.
Definition at line 197 of file batch.py.
◆ build_cmd()
| def build_cmd |
( |
|
self, |
|
|
|
job_id |
|
) |
| |
This is the basic implementation of building a command to run the job from a batch system.
Reimplemented in Auger, Slurm, and LSF.
Definition at line 160 of file batch.py.
◆ default_rundir()
| def default_rundir |
( |
|
self, |
|
|
|
job_id = None |
|
) |
| |
◆ parse_args()
| def parse_args |
( |
|
self, |
|
|
|
args |
|
) |
| |
◆ submit()
This is the generic batch submission function which gets a list of jobs to run based on command line arguments and submits them individually.
It calls the abstract submit_job() method and prints the batch system ID that was returned, if any.
Reimplemented in Pool, Swif, and Auger.
Definition at line 140 of file batch.py.
◆ submit_job()
| def submit_job |
( |
|
self, |
|
|
|
job_id |
|
) |
| |
Submit a single batch job and return the batch ID.
This is abstract as each batch system will do this differently.
Some batch systems don't implement this but sub-classes should override this and make it a no-op so that they can be instantiated.
Reimplemented in Pool, Local, Auger, Slurm, and LSF.
Definition at line 129 of file batch.py.
◆ check_output
◆ config_files
◆ debug
◆ end_job_num
◆ job_ids
◆ job_steps
◆ jobstore
◆ log_dir
◆ parser
◆ run_dir
◆ script
◆ script_name
◆ start_job_num
The documentation for this class was generated from the following file:
1.9.1