Package com.pervasive.datarush.cluster

Provides classes and interfaces related to the management of clustered dataflow engines.

See: Description

Interface Summary

Interface	Description
ClasspathCacheManager
ClasspathCacheManagerImpl.CacheStorageConfiguration
ClusterManager	A central registry for finding the nodes of a cluster.
DistributedWorkerService
FileService
JobMasterService
JobScheduler
MonitoredFutureHandle	Wrapper around the server that handles deletion of results and caching of final result so as to not ping the server once complete.
MonitoredFutureService
NodeAllocationContext
NodeManager	The management interface for a node in a dataflow enabled cluster.
Platform	Interface for container of O/S platform specific values.
WorkerService

Class Summary

Class	Description
CacheScope
CancelResult
ChecksumHelper
ClasspathCacheDirectory
ClasspathCacheManagerImpl
ClasspathCacheManagerImpl.FileSystemCacheStorage
ClasspathChecksumCalculator
ClasspathKey
ClasspathKeyMap
ClasspathSynchronizer
ClientPassThru
ClientWorkerServerArguments
ClusterCLI	Implementation of cluster management command-line interface.
ClusterManagerImpl
ClusterManagerListener
CommandStatus
ConfigurationMacros
ContainerDescriptor	Descriptor of a container (worker) of an external application master.
DataRushCluster	A handle for referencing a distributed DataRush graph engine.
DistributedWorkerServiceImpl
DRFileSystem
DRFileSystemProvider
DRFS
EmbeddedPreferencesProvider
ExecutorOptions	A collection of configuration options for remote execution of graph fragments.
FIFOScheduler
FileData
FileServiceImpl
JobClasspathConfig
JobClasspathConfig.Entry
JobMasterServiceImpl
JobWorkingDirectory
JvmOptions	A collection of configuration options for a jvm
LibraryPathUtil
LogFileOptions
MasterDescriptor	Descriptor of external application masters.
NodeAllocationRequest
NodeConfiguration	Describes the configuration of a NodeManager daemon.
NodeDescriptor	A description of a cluster node.
NodeManagerImpl
NodeMapping
NodeMapping.Entry
NodePartitionPlan
NodeResourceState
NodeUsageProvider
OSGIClasspathSpecifier
OSGIClasspathSpecifier.Element
OSGILauncher
OSGIUtil
PathInfo
PlatformFactory	Obtain the Platform instance for a given O/S architecture.
WorkerServer
WorkerServerArguments

Enum Summary

Enum	Description
CleanupLevel
CommandStatus.State
NodeAllocationStrategy

Exception Summary

Exception	Description
DaemonShutdownException	Exception indicating the requested action could not be performed due to cluster shutdown.

Package com.pervasive.datarush.cluster Description

Provides classes and interfaces related to the management of clustered dataflow engines. The execution of distributed dataflow graphs relies on a properly configured cluster.

A dataflow cluster consists of several pieces:

• A cluster manager, providing a central registry of nodes participating in the cluster
• A node manager, running on each node participating in the cluster
• A fragment executor, launched by the node manager on each node to run graph fragments assigned to the node for a given execution of a distributed graph

The execution of a distributed graph is coordinated by the portion running local to the machine where the graph was started; this portion is referred to as the client graph. When executed the client graph will automatically compose, execute, and monitor the remote fragments. The compilation/execution follows a process like below:

1. The client graph contacts the cluster manager to obtain a list of available nodes and default per-job settings.
2. For each node, the client graph requests a fragment executor to be created using configured per-job settings.
3. Each graph fragment, as determined by the cluster operations defined in the graph, is sent to the appropriate fragment executors to be composed.
4. All graph fragments and the client graph start execution.
5. On completion of the local portions, the client graph waits on all remote fragments to complete. The fragment executors then terminate.