Module datarush.library

Package com.pervasive.datarush.operators

package com.pervasive.datarush.operators
Provides classes and interfaces for developing dataflow operators. Operators represent processing units within a dataflow graph; they are the nodes in the graph. An operator consumes data from its inputs and produces data on its outputs. An alternative, function-oriented view is that an operator is a function which taking some number of flows as arguments and returning some number of flows.

All operators have:

  • An interface defining the inputs and outputs of the operator. Graphs are constructed by linking operators together via their inputs and outputs.
  • Zero or more properties which can be set which control the processing behavior of the operator.

A single operator represents processing to be done on the input data sets. This processing may be done in a single-threaded fashion or in parallel. When executed in parallel, multiple copies of an operator may execute concurrently, each receiving a different portion of the input data and producing a portion of the output data.

Operators may have requirements on the input or may provide guarantees about the output. These are expressed as metadata on the input and output ports. This information is then used by the framework in determining how to execute the dataflow graph.