Modifier and Type | Field and Description |
---|---|
private ADag |
mDag
ADag object corresponding to the Dag whose jobs we want to schedule.
|
protected TransformationStore |
mDAXTransformationStore
Handle to the transformation store that stores the transformation catalog
user specifies in the DAX
|
private java.util.Set |
mExecPools
Set of the execution pools which the user has specified.
|
private SiteSelector |
mSiteSelector
Handle to the site selector.
|
private SLS |
mSLS
The handle to the SLS implementor
|
private Mapper |
mTCMapper
The handle to the transformation catalog mapper object that caches the
queries to the transformation catalog, and indexes them according to
lfn's.
|
private TransformationSelector |
mTXSelector
The handle to the transformation selector, that ends up selecting
what transformations to pick up.
|
private boolean |
mWorkerNodeExecution
A boolean indicating whether to have worker node execution or not.
|
private XMLProducer |
mXMLStore
The XML Producer object that records the actions.
|
static java.lang.String |
REFINER_NAME
The name of the refiner for purposes of error logging
|
mBag, mLogger, mLogMsg, mOutputPool, mPoolFile, mPOptions, mProps, mRLIUrl, mSiteStore, mTCFile, mTCHandle, mTCMode, REGISTRATION_UNIVERSE, TRANSFER_UNIVERSE
Constructor and Description |
---|
InterPoolEngine(ADag aDag,
PegasusBag bag)
Overloaded constructor.
|
InterPoolEngine(PegasusBag bag)
Default constructor.
|
Modifier and Type | Method and Description |
---|---|
java.util.List |
convertToList(java.util.Set s)
Converts a Set to a List.
|
java.util.List |
convertToList(java.util.Vector v)
Converts a Vector to a List.
|
void |
determineSites()
This is where the callout to the Partitioner should take place, that
partitions the workflow into clusters and sends to the site selector only
those list of jobs that are ready to be scheduled.
|
private java.lang.String |
getJobManager(java.lang.String site,
java.lang.String universe)
It returns a jobmanager for the given pool.
|
PegasusBag |
getPegasusBag()
Returns the bag of intialization objects.
|
java.lang.String |
getStagingSite(Job job)
Returns the staging site to be used for a job.
|
ADag |
getWorkflow()
Returns a reference to the workflow that is being refined by the refiner.
|
XMLProducer |
getXMLProducer()
Returns a reference to the XMLProducer, that generates the XML fragment
capturing the actions of the refiner.
|
private void |
handleDependantExecutables(Job job)
Handles the dependant executables that need to be staged.
|
private boolean |
incorporateHint(Job job,
java.lang.String key)
It incorporates a hint in the namespace to the job.
|
private boolean |
incorporateProfiles(Job job)
Incorporates the profiles from the various sources into the job.
|
protected void |
logRefinerAction(Job job)
Logs the action taken by the refiner on a job as a XML fragment in
the XML Producer.
|
void |
scheduleJobs(ADag dag,
java.util.List sites)
It schedules a list of jobs on the execution pools by calling out to the
site selector specified.
|
private TransformationCatalogEntry |
selectTCEntry(java.util.List entries,
Job job,
java.lang.String selector)
Calls out to the transformation selector to select an entry from a list
of valid transformation catalog entries.
|
addVector, appendArrayList, complainForHeadNodeURLPrefix, complainForHeadNodeURLPrefix, loadProperties, printVector, stringInList, stringInPegVector, stringInVector, vectorToString
public static final java.lang.String REFINER_NAME
private ADag mDag
private java.util.Set mExecPools
private SiteSelector mSiteSelector
private TransformationSelector mTXSelector
private Mapper mTCMapper
private XMLProducer mXMLStore
private SLS mSLS
private boolean mWorkerNodeExecution
protected TransformationStore mDAXTransformationStore
public InterPoolEngine(PegasusBag bag)
bag
- the bag of initialization objects.public InterPoolEngine(ADag aDag, PegasusBag bag)
aDag
- the ADag
object corresponding to the Dag
for which we want to determine on which pools to run
the nodes of the Dag.bag
- the bag of initialization objectspublic PegasusBag getPegasusBag()
public ADag getWorkflow()
getWorkflow
in interface Refiner
public XMLProducer getXMLProducer()
getXMLProducer
in interface Refiner
public void determineSites()
public void scheduleJobs(ADag dag, java.util.List sites)
dag
- the abstract workflow.sites
- the list of execution sites, specified by the user.public java.lang.String getStagingSite(Job job)
job
- the job for which to determine the staging siteprivate boolean incorporateProfiles(Job job)
job
- the job into which the profiles have been incorporated.private void handleDependantExecutables(Job job)
job
- Jobprivate TransformationCatalogEntry selectTCEntry(java.util.List entries, Job job, java.lang.String selector)
entries
- list of TransformationCatalogEntry
objects.job
- the job.selectors
- the selector to be calledTransformationCatalogEntry
object
null when transformation selector is unable to select any
transformationprivate java.lang.String getJobManager(java.lang.String site, java.lang.String universe)
site
- the name of the pool.universe
- the universe for which you need the scheduler on that
particular pool.private boolean incorporateHint(Job job, java.lang.String key)
job
- the job that needs the hint to be incorporated.key
- the key in the hint namespace.public java.util.List convertToList(java.util.Vector v)
v
- Vectorpublic java.util.List convertToList(java.util.Set s)
s
- Setprotected void logRefinerAction(Job job)
job
- the Job
containing the job that was mapped
to a site.