hpsmc.batch Namespace Reference

Classes
class	Auger
	Submit Auger batch jobs. More...
class	Batch
	Generic batch processing interface. More...
class	BatchSystem
	Represents a batch processing system that requires submission like Slurm or Auger. More...
class	KillProcessQueue
	Kill processes in the multiprocessing queue if the jobs are canceled. More...
class	Local
	Run local batch jobs sequentially. More...
class	LSF
	Submit LSF batch jobs. More...
class	Pool
	Run a set of jobs in a local multiprocessing pool using Python's multiprocessing module. More...
class	Slurm
	Submit Slurm batch jobs. More...
class	Swif
	Submit jobs to the 'swif2' workflow system at JLAB. More...

Functions
	run_job_pool (cmd)
	Run the command in a new process whose PID is added to a global MP queue.
	is_running (proc)
	Check if a system process looks like it is still running.

Variables
	logger = logging.getLogger("hpsmc.batch")
	RUN_SCRIPT = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'job.py')
	mp_queue = multiprocessing.Queue()
dict	system_dict
	system = sys.argv[1].lower()
dict	batch = system_dict[system]()
	args = sys.argv[2:]

Function Documentation

is_running()

is_running

(

proc

)

Check if a system process looks like it is still running.

Definition at line 1012 of file batch.py.

run_job_pool()

run_job_pool

(

cmd

)

Run the command in a new process whose PID is added to a global MP queue.

Definition at line 997 of file batch.py.

Variable Documentation

args

args = sys.argv[2:]

Definition at line 1137 of file batch.py.

batch

dict batch = system_dict[system]()

Definition at line 1136 of file batch.py.

logger

logger = logging.getLogger("hpsmc.batch")

Definition at line 30 of file batch.py.

mp_queue

mp_queue = multiprocessing.Queue()

Definition at line 994 of file batch.py.

RUN_SCRIPT

RUN_SCRIPT = os.path.join(os.path.dirname(os.path.realpath(__file__)), 'job.py')

Definition at line 32 of file batch.py.

system

system = sys.argv[1].lower()

Definition at line 1133 of file batch.py.

system_dict

dict system_dict

Initial value:

=  {
        "lsf": LSF,
        "slurm": Slurm,
        "auger": Auger,
        "local": Local,
        "pool": Pool,
        "swif": Swif
    }

Definition at line 1124 of file batch.py.