public class CalculateNGramFrequency extends ExecutableOperator implements RecordPipelineOperator
Constructor and Description |
---|
CalculateNGramFrequency()
Default constructor.
|
CalculateNGramFrequency(String textField)
Constructor specifying the tokenized text field to determine the
n-gram frequencies for.
|
CalculateNGramFrequency(String textField,
int n)
Constructor specifying the tokenized text field to determine the
n-gram frequencies for and the n to use.
|
Modifier and Type | Method and Description |
---|---|
protected void |
computeMetadata(StreamingMetadataContext ctx)
Implementations must adhere to the following contracts
|
protected void |
execute(ExecutionContext ctx)
Executes the operator.
|
RecordPort |
getInput()
Get the input port of this operator.
|
String |
getInputField()
Get the field to calculate the n-gram frequencies for.
|
int |
getN()
Get the N for the n-grams.
|
String |
getNgramFreqOutputField()
Get the field that will contain the n-gram frequncy map.
|
RecordPort |
getOutput()
Get the output port of this operator.
|
void |
setInputField(String textField)
Set the field to calculate the n-gram frequencies for.
|
void |
setN(int n)
Set the N for the n-grams.
|
void |
setNgramFreqOutputField(String ngramFreqOutputField)
Set the field that will contain the n-gram frequency map.
|
cloneForExecution, getNumInputCopies, getPortSettings, handleInactiveOutput
disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
disableParallelism, getInputPorts, getOutputPorts
public CalculateNGramFrequency()
setInputField(String)
,
#setNGramFreqOutputField(String)
to set the name
of the text field to count and the output field.public CalculateNGramFrequency(String textField)
textField
- name of the tokenized text field in the inputpublic CalculateNGramFrequency(String textField, int n)
textField
- name of the tokenized text field in the inputn
- the number of words per n-grampublic void setInputField(String textField)
If this field does not exist in the input, or is not of type TokenizedText, an exception will be thrown at composition time.
textField
- name of the tokenized text field in the inputpublic String getInputField()
public void setNgramFreqOutputField(String ngramFreqOutputField)
ngramFreqOutputField
- name of the n-gram field in the outputpublic String getNgramFreqOutputField()
public void setN(int n)
n
- the number of words per n-grampublic int getN()
public RecordPort getInput()
getInput
in interface PipelineOperator<RecordPort>
public RecordPort getOutput()
getOutput
in interface PipelineOperator<RecordPort>
protected void computeMetadata(StreamingMetadataContext ctx)
StreamingOperator
StreamingMetadataContext.parallelize(ParallelismStrategy)
.
RecordPort#setRequiredDataOrdering
, otherwise data may arrive in any order.
RecordPort#setRequiredDataDistribution
, otherwise data will arrive in an unspecified partial distribution
.
RecordPort#getSourceDataDistribution
and RecordPort#getSourceDataOrdering
. These should be
viewed as a hints to help chose a more efficient algorithm. In such cases, though, operators must
still declare data ordering and data distribution requirements; otherwise there is no guarantee that
data will arrive sorted/distributed as required.
RecordPort#setType
.RecordPort#setOutputDataOrdering
RecordPort#setOutputDataDistribution
AbstractModelPort#setMergeHandler
.MergeModel
is a convenient, re-usable model reducer, parameterized with
a merge-handler.
SimpleModelPort
's have no associated metadata and therefore there is
never any output metadata to declare. PMMLPort
's, on the other hand,
do have associated metadata. For all PMMLPorts, implementations must declare
the following:
PMMLPort.setPMMLModelSpec
.
computeMetadata
in class StreamingOperator
ctx
- the contextprotected void execute(ExecutionContext ctx)
ExecutableOperator
execute
in class ExecutableOperator
ctx
- context in which to lookup physical ports bound to logical portsCopyright © 2021 Actian Corporation. All rights reserved.