Module datarush.commons

Package com.pervasive.datarush.cal

Interface DistributedExecutorService

All Known Subinterfaces:

PseudoDistributedExecutorService

All Known Implementing Classes:

com.pervasive.datarush.cal.dr.AbstractDRClusterExecutor, YarnClusterExecutor

public interface DistributedExecutorService

Distributed equivalent of the ExecutorService concept. To obtain a handle to an executor service, use JobExecutorServiceFactory.

Method Summary

Modifier and Type	Method	Description
NodeAllocationPlan	ensureInitialNodeAllocation()	Returns the original node allocation (that which was allocated before any nodes failed).
<T> T	execute(DistributedCallable<T> command, ExecutionOptions options)	Synchronously invokes the given callable.
<T> T	execute(DistributedCallableBatch<T> command, ExecutionOptions options)	Synchronously invokes the given callable.
DistributedCallableContext	getMasterContext()	Returns the job master context.
<T> MonitoredFuture<T>	submit(DistributedCallable<T> callable, ExecutionOptions options)	Asynchronously invokes the given callable.
<T> MonitoredFuture<T>	submit(DistributedCallableBatch<T> callable, ExecutionOptions options)	Asynchronously invokes the given callable.
<T> MonitoredFuture<T>	submitMaster(MonitoredCallable<T> callable)	Asynchronous, local, and non-parallel submission of the given callable.

Method Detail

ensureInitialNodeAllocation

NodeAllocationPlan ensureInitialNodeAllocation()

Returns the original node allocation (that which was allocated before any nodes failed). This method forces allocation of nodes if not already allocated

getMasterContext

DistributedCallableContext getMasterContext()

Returns the job master context. This is the context that nodes will be handled when submitted using ExecutionOptions.MASTER.

Returns:

the non-parallel/non-distributed context

submitMaster

<T> MonitoredFuture<T> submitMaster(MonitoredCallable<T> callable)

Asynchronous, local, and non-parallel submission of the given callable.

Type Parameters:

T - the result type

Parameters:

callable - the callable to run

Returns:

an execution handle

submit

<T> MonitoredFuture<T> submit(DistributedCallable<T> callable,
                              ExecutionOptions options)

Asynchronously invokes the given callable. The command will be invoked once per VirtualNodeID and results/status/exceptions merged together.

NOTE: Implementations are advised to implement this in terms of DistributedCallableBatch. This has the advantage of minimizing communication overhead. It also has the advantage of deserializing the command once-per-JVM which provides command implementations a convenient way of sharing (larger) objects within the same JVM. (DR relies on the assumption for certain performance optimizations).

Type Parameters:

T - the type of results/status

Parameters:

callable - the callable to execute

options - miscellaneous execution options

Returns:

an execution handle

submit

<T> MonitoredFuture<T> submit(DistributedCallableBatch<T> callable,
                              ExecutionOptions options)

Asynchronously invokes the given callable. The command will be invoked once per physicalNodeID and results/status/exceptions merged together.

Type Parameters:

T - the type of results/status

Parameters:

callable - the callable to execute

options - miscellaneous execution options

Returns:

an execution handle

execute

<T> T execute(DistributedCallable<T> command,
              ExecutionOptions options)
       throws DistributedExecutionException,
              InterruptedException

Synchronously invokes the given callable. The command will be invoked once per VirtualNodeID and results/status/exceptions merged together.

NOTE: Implementations are advised to implement this in terms of DistributedCallableBatch. This has the advantage of minimizing communication overhead. It also has the advantage of deserializing the command once-per-JVM which provides command implementations a convenient way of sharing (larger) objects within the same JVM. (DR relies on the assumption for certain performance optimizations).

Type Parameters:

T - the type of results/status

Parameters:

command - the command to execute

options - miscellaneous execution options

Returns:

the result

Throws:

DistributedExecutionException - if an exception occurs during execution

InterruptedException - if interrupted while waiting for results

execute

<T> T execute(DistributedCallableBatch<T> command,
              ExecutionOptions options)
       throws DistributedExecutionException,
              InterruptedException

Synchronously invokes the given callable. The command will be invoked once per physicalNodeID and results/status/exceptions merged together.

Type Parameters:

T - the type of results/status

Parameters:

command - the command to execute

options - miscellaneous execution options

Returns:

an execution handle

Throws:

DistributedExecutionException - if an exception occurs during execution

InterruptedException - if interrupted while waiting for results