com.pervasive.datarush.cluster

Interface ClusterManager

All Superinterfaces:

JobHistoryService

All Known Implementing Classes:

ClusterManagerImpl

public interface ClusterManager
extends JobHistoryService

A central registry for finding the nodes of a cluster. Clients can query the ClusterManager to get information about the cluster, such as the names and management objects of nodes.

End users should not be directly accessing ClusterManager objects; it is public for implementation purposes only.

Method Summary

Methods

Modifier and Type	Method and Description
void	addContainersToAppMaster(JobInformation jobInfo, List<ContainerDescriptor> containers) Add descriptors of containers that are allocated to the application master.
String	allocateMaster(GUID jobGUID) Allocates the master node for the given job
NodeAllocationPlan	allocateWorkers(GUID jobGUID) Allocates the worker nodes for the given job
void	clearClasspathCache() Clears the shared classpath cache
void	deregisterAppMaster(JobInformation jobInfo) Deregister an external application master.
void	ensureRegistered(NodeDescriptor node) Adds a node to the cluster registry.
void	failJob(GUID jobGUID) Manually force job fail
List<MasterDescriptor>	getActiveAppMasters() Get a list of active (external) application masters.
List<JobInformation>	getAllocatedJobs()
ClusterPreferences	getConfiguration() Get the global cluster configuration (not for a specific node).
ClusterPreferences	getConfiguration(String node) Requests the configuration for the given node.
List<ContainerDescriptor>	getContainers(MasterDescriptor masterDescriptor) Get the list of containers allocated to an application master.
ExecutorOptions	getDefaultOptions() Gets the default options configured for remote fragment execution.
Integer	getJobStatus(GUID jobGUID) Get the general run status of the job -1 - job in progress 0 - job completed normally 1+ - job has errors
NodeDescriptor	getNodeDescriptor(String name) Gets the node descriptor for the specified node
NodeMapping	getNodes(boolean refresh) Gets node info for all nodes in the cluster
List<MasterDescriptor>	getPendingAppMasters() Get a list of pending (external) application masters.
List<NodeAllocationRequest>	getPendingJobs()
String	getStatus() Gets a description of the current status of the node
String	getVersion() Returns the version number of this ClusterManager instance
String	initializeNodeAllocation(SerializationBridge<NodeAllocationRequest> request)
void	killJob(GUID jobGUID) Administrative job kill
InputStream	openClusterLogFile(LogFileOptions options) Opens the log file for cluster manager for reading
void	registerAppMaster(MasterDescriptor masterDescriptor) Register an application master started outside of the context of the cluster manager.
void	rotateClusterLog() Rotates the cluster manager log
void	rotateNodeLogs() Rotates all of the node manager logs
void	shutdown() Requests a shutdown of the cluster.
void	shutdownNode(String node) Shuts down the given node.

Methods inherited from interface com.pervasive.datarush.cal.JobHistoryService

jobFinished

Method Detail

getStatus

String getStatus()
                 throws RMIException

Gets a description of the current status of the node

Returns:

a description of the node's status

Throws:

RMIException

getDefaultOptions

ExecutorOptions getDefaultOptions()
                                  throws RMIException

Gets the default options configured for remote fragment execution.

Returns:

the default settings for execution of remote fragments

Throws:

RMIException

getNodes

NodeMapping getNodes(boolean refresh)
                     throws RMIException

Gets node info for all nodes in the cluster

Parameters:

refresh - if true, pings each node to ensure that it only returns live nodes

Returns:

valid node names for the cluster

Throws:

RMIException

getNodeDescriptor

NodeDescriptor getNodeDescriptor(String name)
                                 throws RMIException

Gets the node descriptor for the specified node

Parameters:

name - the name of the node requested

Returns:

the node descriptor

Throws:

RMIException

ensureRegistered

void ensureRegistered(NodeDescriptor node)
                      throws RMIException

Adds a node to the cluster registry. If a node is already registered with the specified name, it is replaced.

Parameters:

node - the node information

nodeMgr - the management object for the node

scratchDir - the scratch directory associated with this node

Throws:

RMIException

shutdownNode

void shutdownNode(String node)
                  throws RMIException

Shuts down the given node.

Parameters:

node - the entry to remove

Throws:

DRException - if the given node is not registered

RMIException

shutdown

void shutdown()
              throws RMIException

Requests a shutdown of the cluster. All registered node managers will also be shutdown.

Throws:

RMIException

getConfiguration

ClusterPreferences getConfiguration(String node)
                                    throws RMIException

Requests the configuration for the given node.

Parameters:

node - the node requesting the configuration

Returns:

the configuration for the given node

Throws:

RMIException

openClusterLogFile

InputStream openClusterLogFile(LogFileOptions options)
                               throws RMIException,
                                      IOException

Opens the log file for cluster manager for reading

Parameters:

options -

Returns:

stream to read the log file

Throws:

IOException

RMIException

killJob

void killJob(GUID jobGUID)
             throws RMIException

Administrative job kill

Parameters:

jobGUID -

Throws:

RMIException

failJob

void failJob(GUID jobGUID)
             throws RMIException

Manually force job fail

Parameters:

jobGUID -

Throws:

RMIException

getJobStatus

Integer getJobStatus(GUID jobGUID)
                     throws RMIException

Get the general run status of the job -1 - job in progress 0 - job completed normally 1+ - job has errors

Parameters:

jobGUID -

Returns:

integer describing run status

Throws:

RMIException

clearClasspathCache

void clearClasspathCache()
                         throws RMIException,
                                IOException

Clears the shared classpath cache

Throws:

IOException - if unable to delete (would typically happen if job is running). Generally we don't care so much about cleanup failures, but here we need to let the user know because otherwise case may still be dirty.

RMIException

rotateClusterLog

void rotateClusterLog()
                      throws RMIException

Rotates the cluster manager log

Throws:

RMIException

rotateNodeLogs

void rotateNodeLogs()
                    throws RMIException,
                           IOException

Rotates all of the node manager logs

Throws:

RMIException

IOException - if unable to rotate

getVersion

String getVersion()

Returns the version number of this ClusterManager instance

Returns:

the version

initializeNodeAllocation

String initializeNodeAllocation(SerializationBridge<NodeAllocationRequest> request)

allocateMaster

String allocateMaster(GUID jobGUID)

Allocates the master node for the given job

Parameters:

jobInformation -

Returns:

allocateWorkers

NodeAllocationPlan allocateWorkers(GUID jobGUID)

Allocates the worker nodes for the given job

Parameters:

jobGUID -

resources -

Returns:

the node allocation plan

getAllocatedJobs

List<JobInformation> getAllocatedJobs()

getPendingJobs

List<NodeAllocationRequest> getPendingJobs()

getConfiguration

ClusterPreferences getConfiguration()
                                    throws RMIException

Get the global cluster configuration (not for a specific node).

Returns:

global configuration

Throws:

RMIException - thrown for communication errors

registerAppMaster

void registerAppMaster(MasterDescriptor masterDescriptor)
                       throws RMIException

Register an application master started outside of the context of the cluster manager.

Parameters:

masterDescriptor - descriptor of the application master

Throws:

RMIException - thrown for communication errors

addContainersToAppMaster

void addContainersToAppMaster(JobInformation jobInfo,
                            List<ContainerDescriptor> containers)
                              throws RMIException

Add descriptors of containers that are allocated to the application master. This may be called more than once per application master.

Parameters:

masterDescriptor - application master

containers - containers allocated to the application master

Throws:

RMIException - thrown for communication errors

deregisterAppMaster

void deregisterAppMaster(JobInformation jobInfo)
                         throws RMIException

Deregister an external application master. The status of the application will no longer be tracked by the cluster manager.

Parameters:

masterDescriptor - application master to de-list

Throws:

RMIException - thrown for communication errors

getPendingAppMasters

List<MasterDescriptor> getPendingAppMasters()
                                            throws RMIException

Get a list of pending (external) application masters. A pending application master is up and running, but is awaiting worker container resources.

Returns:

pending application masters

Throws:

RMIException - throw for communication errors

getActiveAppMasters

List<MasterDescriptor> getActiveAppMasters()
                                           throws RMIException

Get a list of active (external) application masters. Active application masters have allocated resources and running containers.

Returns:

active application masters

Throws:

RMIException - throw for communication errors

getContainers

List<ContainerDescriptor> getContainers(MasterDescriptor masterDescriptor)
                                        throws RMIException

Get the list of containers allocated to an application master.

Parameters:

masterDescriptor - application master

Returns:

list of containers allocated to the application master

Throws:

RMIException - throws for communication errors