Module datarush.library
Package com.pervasive.datarush.operators

Class CompositeOperator

java.lang.Object
com.pervasive.datarush.operators.AbstractLogicalOperator
com.pervasive.datarush.operators.CompositeOperator
All Implemented Interfaces:
LogicalOperator
Direct Known Subclasses:
AbstractReader, AbstractRecordCompositeOperator, AbstractRelationalJoin, AbstractWriter, AnalyzeDuplicateKeys, AnalyzeLinkKeys, AssertEqualHash, AssertEqualRecordType, AssertEqualTypes, AssertRowCount, BlockCartesian, BlockRecords, BlockSelf, ClusterDuplicates, ClusterLinks, DataQualityAnalyzer, DiscoverDomain, DiscoverDuplicates, DiscoverLinks, DrawDiagnosticsChart, DumpPartitions, FPGrowth, FrequentItems, JDBCOperator, KeyOperator, KNNClassifier, LoadActianVector, LogisticRegressionLearner, NaiveBayesLearner, OpenComposite, OpenModelSink, OpenModelSource, OpenMultiModelSink, OpenMultiModelSource, OpenMultiRecordSink, OpenMultiRecordSource, OpenRecordSink, OpenRecordSource, ReadActianVector, ReadPMML, ReadSource, RemoveDuplicates, ReplaceMissingValues, RowsToColumns, RunRScript, SummaryStatistics
public abstract class CompositeOperator extends AbstractLogicalOperator
To be implemented by operators that can be defined by chaining together other operations. CompositeOperators are commonly used in cases where an overall operation can be defined by a mixture parallelizable and non-parallelizable operations or a mixture of operations that have different distribution requirements. An example of this parallel/non-parallel mix would be the Group operator: in the case of a key-less aggregation, there is an initial parallel operator where partial aggregations are computed connected to a non-parallel operator that combines the one-row from each partition into a final result.

Here is an example of a CompositeOperator that consists of two sub-operators, linked together: 

    public class MyComposite extends CompositeOperator {
       //the input port for the composite
       private final RecordPort input= newRecordInput("input");
       //the output port for the composite
       private final RecordPort output= newRecordOutput("output");
       
       public MyComposite() {
       }
       
       //the input port for the composite
       public RecordPort getInput() {
          return input;
       }
       
       //the output port for the composite
       public RecordPort getOutput() {
          return output;
       }

       @Override
       protected void compose(CompositionContext ctx) {
           //perform any validation
           if (...) {
              throw new ...;
           }
       
           //add the two sub-operators
           SubOp1 op1= ctx.add(new SubOp1());
           SubOp2 op2= ctx.add(new SubOp2());
           
           //connect the composite's input port to the op1's input
           ctx.connect(input,op1.getInput());
           //connect op1's output to op2's input
           ctx.connect(op1.getOutput(), op2.getInput());
           //connect op2's output to the composite's output port
           ctx.connect(op2.getOutput(), output);
       }
    }

  • Constructor Details

    • CompositeOperator

      public CompositeOperator()

  • Method Details

    • compose

      protected abstract void compose(CompositionContext ctx)
      Compose the body of this operator. Implementations should do the following:
      1. Perform any validation of configuration, input types, etc
      2. Instantiate and configure sub-operators, adding them to the provided context via the method OperatorComposable.add(O)
      3. Create necessary connections via the method OperatorComposable.connect(P, P). This includes connections from the composite's input ports to sub-operators, connections between sub-operators, and connections from sub-operators output ports to the composite's output ports
      Parameters:
      ctx - the context