|
HPS-MC
|
|
|
HPS-MC
|
|
Primary class to run HPS jobs from a Python script. More...
Public Member Functions | |
| __init__ (self, args=sys.argv, **kwargs) | |
| add (self, component) | |
| Public method for adding components to the job. | |
| set_parameters (self, params) | |
| Add parameters to the job, overriding values if they exist already. | |
| parse_args (self) | |
| Configure the job from command line arguments. | |
| run (self) | |
| This is the primary execution method for running the job. | |
| ptag (self, tag, filename) | |
| Map a key to an output file name so a user can reference it in their job params. | |
| resolve_output_src (self, src) | |
Static Public Member Functions | |
| is_ptag (src) | |
| get_ptag_from_src (src) | |
Public Attributes | |
| hpsmc_dir | |
| args | |
| (passed) job arguments | |
| job_config | |
| Job configuration. | |
| description | |
| short description of job, should be overridden by the job script | |
| job_id | |
| job ID | |
| param_file | |
| path to parameter file | |
| components | |
| list of components in job | |
| rundir | |
| rundir is current working directory | |
| params | |
| dict of parameters | |
| output_dir | |
| output_dir is current working directory | |
| input_files | |
| dict of input files | |
| output_files | |
| dict of output files | |
| ptags | |
| dict with keys to output filenames | |
| component_out | |
| output for component printouts | |
| component_err | |
| output for component error messages | |
| script | |
| script containing component initializations | |
| job_steps | |
| job steps | |
| hps_fieldmaps_dir | |
| fieldmaps dir | |
| enable_copy_output_files | |
| These attributes can all be set in the config file. | |
| enable_copy_input_files | |
| enable_cleanup | |
| delete_existing | |
| delete_rundir | |
| dry_run | |
| ignore_return_codes | |
| check_output_files | |
| check_commands | |
| enable_file_chaining | |
| enable_env_config | |
| out | |
| err | |
Static Public Attributes | |
| str | PTAG_PREFIX = 'ptag:' |
Protected Member Functions | |
| _load_params (self, params) | |
| Load the job parameters from JSON data. | |
| _set_input_files (self) | |
| Prepare dictionary of input files. | |
| _initialize (self) | |
| Perform basic initialization before the job script is loaded. | |
| _configure (self) | |
| Configure job class and components. | |
| _load_script (self) | |
| Load the job script. | |
| _execute (self) | |
| Execute all components in job. | |
| _setup (self) | |
| Necessary setup before job can be executed. | |
| _config_file_pipeline (self) | |
| Pipe component outputs to inputs automatically. | |
| _set_parameters (self) | |
| Push JSON job parameters to components. | |
| _cleanup (self) | |
| Perform post-job cleanup. | |
| _copy_output_files (self) | |
| Copy output files to output directory, handling ptags if necessary. | |
| _copy_output_file (self, src, dest) | |
| Copy an output file from src to dest. | |
| _copy_input_files (self) | |
| Copy input files to the run dir. | |
| _symlink_input_files (self) | |
| Symlink input files. | |
| _config_fieldmap_dir (self) | |
| Set fieldmap dir to install location if not provided in config. | |
| _symlink_fieldmap_dir (self) | |
| Symlink to the fieldmap directory. | |
Primary class to run HPS jobs from a Python script.
Jobs are run by executing a series of components which are configured using a config file with parameters provided by a JSON job file.
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
|
protected |
Prepare dictionary of input files.
If a link to a download location is given as input, the file is downloaded into the run directory before the file name is added to the input_files dict. If a regular file is provided, it is added to the dict without any additional action.
|
protected |
|
protected |
|
protected |
| add | ( | self, | |
| component | |||
| ) |
| parse_args | ( | self | ) |
| ptag | ( | self, | |
| tag, | |||
| filename | |||
| ) |
| run | ( | self | ) |
| set_parameters | ( | self, | |
| params | |||
| ) |
| description |
| enable_copy_output_files |
|
static |
| rundir |
| script |
1.9.8