Module datarush.library

Package com.pervasive.datarush.operators

Interface StreamingMetadataContext

All Superinterfaces:

MetadataCalculationContext, MetadataContext

public interface StreamingMetadataContext
extends MetadataCalculationContext

Context used when computing metadata via StreamingOperator.computeMetadata(StreamingMetadataContext). This provides methods for resolving/computing metadata.

See Also:

StreamingOperator.computeMetadata(StreamingMetadataContext)

Method Summary

Modifier and Type	Method	Description
void	parallelize(ParallelismStrategy strategy)	Controls the parallelism of this operator.
void	setClientExecution(boolean clientExecution)	Sets a flag indicating whether client execution is required.
void	setParallelizable(boolean parallelizable)	Deprecated. Use parallelize(ParallelismStrategy)

Methods inherited from interface com.pervasive.datarush.operators.MetadataCalculationContext

getCompilationLevel, getOperator, setOutputMetadata, setOutputMetadataDynamic, setRequiredMetadata, setStagingForced

Methods inherited from interface com.pervasive.datarush.operators.MetadataContext

getCombinedMetadata, getEngineConfig, getFileClient, getMaxParallelism, getPath, getRequiredMetadata, getSourceMaxParallelism, getSourceMetadata, isParallel, isSourceConnected, isSourceParallel

Method Detail

parallelize

void parallelize(ParallelismStrategy strategy)

Controls the parallelism of this operator. Is ParallelismStrategy.NON_PARALLELIZABLE by default. If non-parallelizable, all data will be brought to a single processor on the machine/cluster. In general, most operations must be parallelizable for them to be scalable. Some operations (composites) may consist of parallelizable operations that operator on large pieces of data, followed by non-parallelizable operations that operate on smaller pieces.

Parameters:

strategy - the strategy to use to determine parallelization.

See Also:

ParallelismStrategy

setParallelizable

void setParallelizable(boolean parallelizable)

Deprecated.

Use parallelize(ParallelismStrategy)

Sets whether to parallelize. A value of {@link true} is equivalent to calling parallelize(ParallelismStrategy.CONFIGURED). A value of false is equivalent to calling parallelize(ParallelismStrategy.NON_PARALLELIZABLE)

Parameters:

parallelizable -

setClientExecution

void setClientExecution(boolean clientExecution)

Sets a flag indicating whether client execution is required. This is only supported for sources and sinks. Note that "client" is a relative term, referring to the JVM in which the graph or subgraph was composed. In the case of a top-level graph, "client" is the machine the originated the graph. In the case of an IterativeOperator, refers to the JVM that composed the subgraph (in the case of clustered execution, this will be the job master).

Parameters:

clientExecution - whether client execution is required.

Throws:

UnsupportedOperationException - if the current node is neither a source nor a sink.