public class UnionAll extends DeferredCompositeOperator implements SourceOperator<RecordPort>
The type of the output is determined by setting the union mode. If the outputMapping setting is set to MAPBYSCHEMA then a schema must be provided which will define the output type. Otherwise the output type can be automatically determined by setting MAPBYPOSITION or MAPBYNAME, which will determine an appropriate output type by mapping the two inputs by position or name respectively.
In the case where a target schema is provided the input fields are mapped to output fields based on field name. If a field in the output is not contained in the target schema, the field is dropped. If a field is contained in the output schema, but not in the input, the output field will contain NULL values. Input values are converted into the specified output field type where possible.
For example, if the left input contains fields {a:int, b:int, c:string} and the right input contains fields {a:long, b:double, d:string} and the target schema specifies fields {a:double, c:string, d:date, e:string} then the following is true:
integer
type to a double
type.long
type to a double
type.string
type to a date
type.
The format pattern specified in the target schema will be used for the conversion, if specified.In the case where a target schema is not provided, the two input ports must have compatible types. In this case the operator will try to determine valid output fields based on the left and right input and two settings. The outputMapping setting can be set to MAPBYNAME if the left and right side should be matched by field name. Otherwise the operator can use MAPBYPOSITION and they will be matched by position. Additionally the allowExtraFields setting can be set to true if fields that are only present on one side of the input should be retained. If the field is not present in one of the inputs it will contain NULL values. If extra fields are present in one of the inputs and allowExtraFields is set to false an error will be thrown.
Modifier and Type | Class and Description |
---|---|
static class |
UnionAll.UnionMode |
Constructor and Description |
---|
UnionAll() |
UnionAll(RecordTextSchema<?> targetSchema) |
Modifier and Type | Method and Description |
---|---|
protected void |
compose(DeferredCompositionContext ctx)
Compose the body of this operator.
|
protected void |
computeMetadata(StreamingMetadataContext ctx)
This operator can execute in parallel.
|
static RecordTokenType |
generateSchema(RecordTokenType leftType,
RecordTokenType rightType,
UnionAll.UnionMode mode,
boolean keepExtraFields)
Generate a schema for the union of two records.
|
boolean |
getIncludeExtraFields()
Will be true if the generated schema will include unmapped fields from either side.
|
RecordPort |
getLeft()
Returns the left input port.
|
RecordPort |
getOutput()
Returns the output port.
|
UnionAll.UnionMode |
getOutputMapping()
Get how the output type should be determined.
|
RecordPort |
getRight()
Returns the right input port.
|
RecordTextSchema<?> |
getSchema()
Gets the target record schema defining the output type.
|
void |
setIncludeExtraFields(boolean includeExtraFields)
Set to true if the generated schema should include unmapped field from either side.
|
void |
setOutputMapping(UnionAll.UnionMode outputMapping)
Set how the output type should be determined.
|
void |
setSchema(RecordTextSchema<?> schema)
Sets the optional target output schema.
|
computeOutputTypes
disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
disableParallelism, getInputPorts, getOutputPorts
public UnionAll()
public UnionAll(RecordTextSchema<?> targetSchema)
public final RecordPort getLeft()
public final RecordPort getRight()
public final RecordPort getOutput()
getOutput
in interface SourceOperator<RecordPort>
public RecordTextSchema<?> getSchema()
public void setSchema(RecordTextSchema<?> schema)
schema
- the record schema defining the output typepublic UnionAll.UnionMode getOutputMapping()
public void setOutputMapping(UnionAll.UnionMode outputMapping)
outputMapping
- the union mapping modepublic boolean getIncludeExtraFields()
public void setIncludeExtraFields(boolean includeExtraFields)
includeExtraFields
- if unmapped fields will be included in the generated schemaprotected void computeMetadata(StreamingMetadataContext ctx)
computeMetadata
in class DeferredCompositeOperator
ctx
- the contextpublic static RecordTokenType generateSchema(RecordTokenType leftType, RecordTokenType rightType, UnionAll.UnionMode mode, boolean keepExtraFields)
leftType
- the left record typerightType
- the right record typemode
- the union modekeepExtraFields
- if true fields only present on one side of the input will be retainedprotected void compose(DeferredCompositionContext ctx)
DeferredCompositeOperator
OperatorComposable.add(O)
OperatorComposable.connect(P, P)
. This includes
connections from the composite's input ports to sub-operators, connections between sub-operators, and
connections from sub-operators output ports to the composite's output portscompose
in class DeferredCompositeOperator
ctx
- the contextCopyright © 2016 Actian Corporation. All rights reserved.