Module datarush.hadoop.core
Package com.actian.dataflow.cal.yarn.client

Class YarnClusterJobExecutor

java.lang.Object
com.pervasive.datarush.cal.dr.AbstractDRJobExecutor
com.actian.dataflow.cal.yarn.client.YarnClusterJobExecutor
All Implemented Interfaces:
JobExecutorService
public class YarnClusterJobExecutor extends com.pervasive.datarush.cal.dr.AbstractDRJobExecutor
A job executor is created for each new job. The executor wraps several pieces of needed information including the file client, job master (started with the start(FileClient, JobSpecifier) method), and the job specifier (which itself contains the classpath specifier, job info, requested resources, and storage management path for the job).

  • Nested Class Summary

    Nested classes/interfaces inherited from class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

    com.pervasive.datarush.cal.dr.AbstractDRJobExecutor.JobMasterHelper

  • Field Summary

    Fields inherited from class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

    fileClient, jobSpec, MUTEX

  • Constructor Summary

    Constructors
    Constructor
    Description
    YarnClusterJobExecutor(FileClient fileClient, JobSpecifier jobSpec)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    StorageManager
    getExternalStorageManager()
    Has the side effect of starting the application master.
    protected com.pervasive.datarush.cal.dr.AbstractDRJobExecutor.JobMasterHelper
    start(FileClient fileClient, JobSpecifier jobSpec)
     

    Methods inherited from class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor

    ensureJobMaster, getJobInformation, startShutdown, submit

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

  • Method Details

    • getExternalStorageManager

      public StorageManager getExternalStorageManager()
      Has the side effect of starting the application master.
      Returns:
      the storage manager

    • start

      protected com.pervasive.datarush.cal.dr.AbstractDRJobExecutor.JobMasterHelper start(FileClient fileClient, JobSpecifier jobSpec)
      Specified by:
      start in class com.pervasive.datarush.cal.dr.AbstractDRJobExecutor