|
HPS-MC
|
|
|
HPS-MC
|
|
Primary class to run HPS jobs from a Python script. More...
Public Member Functions | |
| def | __init__ (self, args=sys.argv, **kwargs) |
| def | add (self, component) |
| Public method for adding components to the job. More... | |
| def | set_parameters (self, params) |
| Add parameters to the job, overriding values if they exist already. More... | |
| def | parse_args (self) |
| Configure the job from command line arguments. More... | |
| def | run (self) |
| This is the primary execution method for running the job. More... | |
| def | ptag (self, tag, filename) |
| Map a key to an output file name so a user can reference it in their job params. More... | |
| def | resolve_output_src (self, src) |
Static Public Member Functions | |
| def | is_ptag (src) |
| def | get_ptag_from_src (src) |
Public Attributes | |
| hpsmc_dir | |
| args | |
| (passed) job arguments More... | |
| job_config | |
| Job configuration. More... | |
| description | |
| short description of job, should be overridden by the job script More... | |
| job_id | |
| job ID More... | |
| param_file | |
| path to parameter file More... | |
| components | |
| list of components in job More... | |
| rundir | |
| rundir is current working directory More... | |
| params | |
| dict of parameters More... | |
| output_dir | |
| output_dir is current working directory More... | |
| input_files | |
| dict of input files More... | |
| output_files | |
| dict of output files More... | |
| ptags | |
| dict with keys to output filenames More... | |
| component_out | |
| output for component printouts More... | |
| component_err | |
| output for component error messages More... | |
| script | |
| script containing component initializations More... | |
| job_steps | |
| job steps More... | |
| hps_fieldmaps_dir | |
| fieldmaps dir More... | |
| enable_copy_output_files | |
| These attributes can all be set in the config file. More... | |
| enable_copy_input_files | |
| delete_existing | |
| delete_rundir | |
| dry_run | |
| ignore_return_codes | |
| check_output_files | |
| check_commands | |
| enable_file_chaining | |
| enable_env_config | |
| out | |
| err | |
Static Public Attributes | |
| string | PTAG_PREFIX = 'ptag:' |
Private Member Functions | |
| def | _load_params (self, params) |
| Load the job parameters from JSON data. More... | |
| def | _set_input_files (self) |
| Prepare dictionary of input files. More... | |
| def | _initialize (self) |
| Perform basic initialization before the job script is loaded. More... | |
| def | _configure (self) |
| Configure job class and components. More... | |
| def | _load_script (self) |
| Load the job script. More... | |
| def | _execute (self) |
| Execute all components in job. More... | |
| def | _setup (self) |
| Necessary setup before job can be executed. More... | |
| def | _config_file_pipeline (self) |
| Pipe component outputs to inputs automatically. More... | |
| def | _set_parameters (self) |
| Push JSON job parameters to components. More... | |
| def | _cleanup (self) |
| Perform post-job cleanup. More... | |
| def | _copy_output_files (self) |
| Copy output files to output directory, handling ptags if necessary. More... | |
| def | _copy_output_file (self, src, dest) |
| Copy an output file from src to dest. More... | |
| def | _copy_input_files (self) |
| Copy input files to the run dir. More... | |
| def | _symlink_input_files (self) |
| Symlink input files. More... | |
| def | _config_fieldmap_dir (self) |
| Set fieldmap dir to install location if not provided in config. More... | |
| def | _symlink_fieldmap_dir (self) |
| Symlink to the fieldmap directory. More... | |
Primary class to run HPS jobs from a Python script.
Jobs are run by executing a series of components which are configured using a config file with parameters provided by a JSON job file.
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
|
private |
Prepare dictionary of input files.
If a link to a download location is given as input, the file is downloaded into the run directory before the file name is added to the input_files dict. If a regular file is provided, it is added to the dict without any additional action.
|
private |
|
private |
|
private |
|
private |
| def add | ( | self, | |
| component | |||
| ) |
| def parse_args | ( | self | ) |
| def ptag | ( | self, | |
| tag, | |||
| filename | |||
| ) |
| def run | ( | self | ) |
| def set_parameters | ( | self, | |
| params | |||
| ) |
| description |
| enable_copy_output_files |
| out |
The files should be symlinked if copying is disabled. parser.add_argument("--no-copy-output-files", help="Disable copying of output files") parser.add_argument('–feature', dest='feature', action='store_true') parser.add_argument('–no-feature', dest='feature', action='store_false') parser.set_defaults(feature=True)
|
static |
| script |
1.9.1