public abstract class AbstractDRClusterExecutor extends Object implements DistributedExecutorService
Modifier and Type | Class and Description |
---|---|
protected static class |
AbstractDRClusterExecutor.CommandServiceHelper |
protected static class |
AbstractDRClusterExecutor.CommandServiceInfo |
Modifier and Type | Field and Description |
---|---|
protected NodeAllocationProvider |
allocationProvider |
protected Object |
DIST_MUTEX |
protected List<AbstractDRClusterExecutor.CommandServiceInfo> |
distExecutors |
protected FileClient |
fileClient |
protected JobSpecifier |
jobSpec |
Constructor and Description |
---|
AbstractDRClusterExecutor(FileClient fileClient,
JobSpecifier jobSpec,
NodeAllocationProvider allocationProvider) |
Modifier and Type | Method and Description |
---|---|
protected abstract <R> MonitoredFuture<R> |
createFuture(DistributedCallableBatch<R> batch,
List<CommandHandleInfo> handles,
ExecutionOptions options) |
Map<String,Path> |
drfsRoots() |
NodeAllocationPlan |
ensureInitialNodeAllocation()
Returns the original node allocation (that which was allocated before any nodes failed).
|
<T> T |
execute(DistributedCallable<T> command,
ExecutionOptions options)
Synchronously invokes the given callable.
|
<T> T |
execute(DistributedCallableBatch<T> command,
ExecutionOptions options)
Synchronously invokes the given callable.
|
protected List<AbstractDRClusterExecutor.CommandServiceInfo> |
fullPartitions() |
DistributedCallableContext |
getMasterContext()
Returns the job master context.
|
protected ResourceAllocation |
getMasterResources() |
NodeStatistics |
getNonParallelStatistics()
Returns the non-parallel stats ( these are treated as the stats for the master )
|
protected abstract List<AbstractDRClusterExecutor.CommandServiceInfo> |
start() |
List<NodeStatistics> |
startShutdown() |
<R> MonitoredFuture<R> |
submit(DistributedCallable<R> command,
ExecutionOptions options)
Asynchronously invokes the given callable.
|
<R> MonitoredFuture<R> |
submit(DistributedCallableBatch<R> command,
ExecutionOptions options)
Asynchronously invokes the given callable.
|
protected abstract <R> List<CommandHandleInfo> |
submit(DistributedCallableBatch<R> command,
List<AbstractDRClusterExecutor.CommandServiceHelper> services)
Overridden to either do parallel submit (for dist) or non-parallel (for pseudo-dist)
|
<T> MonitoredFuture<T> |
submitMaster(MonitoredCallable<T> command)
Asynchronous, local, and non-parallel submission of the given callable.
|
protected final FileClient fileClient
protected final JobSpecifier jobSpec
protected final NodeAllocationProvider allocationProvider
protected final Object DIST_MUTEX
protected List<AbstractDRClusterExecutor.CommandServiceInfo> distExecutors
public AbstractDRClusterExecutor(FileClient fileClient, JobSpecifier jobSpec, NodeAllocationProvider allocationProvider)
public final NodeAllocationPlan ensureInitialNodeAllocation()
DistributedExecutorService
ensureInitialNodeAllocation
in interface DistributedExecutorService
public final NodeStatistics getNonParallelStatistics()
protected ResourceAllocation getMasterResources()
protected final List<AbstractDRClusterExecutor.CommandServiceInfo> fullPartitions()
protected abstract List<AbstractDRClusterExecutor.CommandServiceInfo> start()
public final DistributedCallableContext getMasterContext()
DistributedExecutorService
ExecutionOptions.MASTER
.getMasterContext
in interface DistributedExecutorService
public final <T> MonitoredFuture<T> submitMaster(MonitoredCallable<T> command)
DistributedExecutorService
submitMaster
in interface DistributedExecutorService
T
- the result typecommand
- the callable to runpublic final List<NodeStatistics> startShutdown()
public final <R> MonitoredFuture<R> submit(DistributedCallable<R> command, ExecutionOptions options)
DistributedExecutorService
VirtualNodeID
and
results/status/exceptions merged together.
NOTE: Implementations are advised to implement this in terms of DistributedCallableBatch. This has the
advantage of minimizing communication overhead. It also has the advantage of deserializing
the command once-per-JVM which provides command implementations a convenient way of sharing (larger) objects
within the same JVM. (DR relies on the assumption for certain performance optimizations).submit
in interface DistributedExecutorService
R
- the type of results/statuscommand
- the callable to executeoptions
- miscellaneous execution optionspublic final <R> MonitoredFuture<R> submit(DistributedCallableBatch<R> command, ExecutionOptions options)
DistributedExecutorService
physicalNodeID
and
results/status/exceptions merged together.
submit
in interface DistributedExecutorService
R
- the type of results/statuscommand
- the callable to executeoptions
- miscellaneous execution optionsprotected abstract <R> List<CommandHandleInfo> submit(DistributedCallableBatch<R> command, List<AbstractDRClusterExecutor.CommandServiceHelper> services)
command
- services
- public final <T> T execute(DistributedCallable<T> command, ExecutionOptions options) throws DistributedExecutionException, InterruptedException
DistributedExecutorService
VirtualNodeID
and
results/status/exceptions merged together.
NOTE: Implementations are advised to implement this in terms of DistributedCallableBatch. This has the
advantage of minimizing communication overhead. It also has the advantage of deserializing
the command once-per-JVM which provides command implementations a convenient way of sharing (larger) objects
within the same JVM. (DR relies on the assumption for certain performance optimizations).execute
in interface DistributedExecutorService
T
- the type of results/statuscommand
- the command to executeoptions
- miscellaneous execution optionsDistributedExecutionException
- if an exception occurs during executionInterruptedException
- if interrupted while waiting for resultspublic final <T> T execute(DistributedCallableBatch<T> command, ExecutionOptions options) throws DistributedExecutionException, InterruptedException
DistributedExecutorService
physicalNodeID
and
results/status/exceptions merged together.
execute
in interface DistributedExecutorService
T
- the type of results/statuscommand
- the command to executeoptions
- miscellaneous execution optionsDistributedExecutionException
- if an exception occurs during executionInterruptedException
- if interrupted while waiting for resultsprotected abstract <R> MonitoredFuture<R> createFuture(DistributedCallableBatch<R> batch, List<CommandHandleInfo> handles, ExecutionOptions options)
Copyright © 2016 Actian Corporation. All Rights Reserved.