Module datarush.hadoop.core

Package com.actian.dataflow.cal.yarn.client

Class YarnClusterJobExecutor

java.lang.Object

com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

com.actian.dataflow.cal.yarn.client.YarnClusterJobExecutor

All Implemented Interfaces:

JobExecutorService

public class YarnClusterJobExecutor
extends com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

A job executor is created for each new job. The executor wraps several pieces of needed information including the file client, job master (started with the start(FileClient, JobSpecifier) method), and the job specifier (which itself contains the classpath specifier, job info, requested resources, and storage management path for the job).

Nested Class Summary

Nested classes/interfaces inherited from class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

com.pervasive.datarush.cal.dr.AbstractDRJobExecutor.JobMasterHelper

Field Summary

Fields inherited from class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

fileClient, jobSpec, MUTEX

Constructor Summary

Constructors

Constructor	Description
YarnClusterJobExecutor(FileClient fileClient, JobSpecifier jobSpec)

Method Summary

Modifier and Type	Method	Description
StorageManager	getExternalStorageManager()	Has the side effect of starting the application master.
protected com.pervasive.datarush.cal.dr.AbstractDRJobExecutor.JobMasterHelper	start(FileClient fileClient, JobSpecifier jobSpec)

Methods inherited from class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

ensureJobMaster, getJobInformation, startShutdown, submit

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

YarnClusterJobExecutor

public YarnClusterJobExecutor(FileClient fileClient,
                              JobSpecifier jobSpec)

Method Detail

getExternalStorageManager

public StorageManager getExternalStorageManager()

Has the side effect of starting the application master.

Returns:

the storage manager

start

protected com.pervasive.datarush.cal.dr.AbstractDRJobExecutor.JobMasterHelper start(FileClient fileClient,
                                                                                    JobSpecifier jobSpec)

Specified by:

start in class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor