public abstract class AbstractWriter extends CompositeOperator implements RecordSinkOperator
ByteSink
.
This is most often a file, so convenience methods
for specifying the target as a file are provided.AbstractWriter
wraps a WriteSink
operator.
Implementations provide an interface specific to
an appropriate class of files - delimited text files, as an
example - hiding the more complex model used to describe
data parsing in general. The composition structure is the
same for all readers with only DataFormat
used
differing between implementations.Modifier and Type | Field and Description |
---|---|
protected RecordPort |
input
The input port of the read operator
|
protected FormattingOptions |
options
Container for options related to formatting
|
Modifier | Constructor and Description |
---|---|
protected |
AbstractWriter()
Writes an empty target with default settings.
|
protected |
AbstractWriter(boolean includeDoneSignal)
Writes an empty target with default settings,
optionally providing a port for signaling
completion of the write.
|
protected |
AbstractWriter(ByteSink target,
WriteMode mode)
Writes to the specified target sink in the given
mode.
|
protected |
AbstractWriter(Path path,
WriteMode mode)
Writes to the specified path in the given mode.
|
protected |
AbstractWriter(String path,
WriteMode mode)
Writes to the specified path in the given mode.
|
Modifier and Type | Method and Description |
---|---|
protected void |
compose(CompositionContext ctx)
Composes a writer for the target, using configured options
and a derived format.
|
protected abstract DataFormat |
computeFormat(CompositionContext ctx)
Determines the data format for the target.
|
FormattingOptions |
getFormatOptions()
Gets the formatting options used by the writer.
|
RecordPort |
getInput()
Gets the record port providing the records to write to the target sink.
|
WriteMode |
getMode()
Gets how existing files should be handled
by the writer.
|
boolean |
getSaveMetadata()
If set to true, the writer will attempt to save the metadata with the data
if the implementation supports it.
|
ByteSink |
getTarget()
Gets the data sink for the writer.
|
int |
getWriteBuffer()
Gets the size of the I/O buffer, in bytes,
to use for writes.
|
boolean |
getWriteOnClient()
Indicates whether the writer should write a file on the client.
|
boolean |
getWriteSingleSink()
Indicates whether the writer produces a
single output file.
|
boolean |
isIgnoreSortOrder()
If set to true, the writer will write in any order.
|
void |
setFormatOptions(FormattingOptions options)
Sets the formatting options used by the reader.
|
void |
setIgnoreSortOrder(boolean ignoreSortOrder)
If set to true, the writer will write in any order.
|
void |
setMode(WriteMode mode)
Sets how the writer should handle an existing
target.
|
void |
setSaveMetadata(boolean saveMetadata)
If set to true, the writer will attempt to save the metadata with the data
if the implementation supports it.
|
void |
setTarget(ByteSink sink)
Sets the data sink for the writer.
|
void |
setTarget(Path path)
Sets the data sink for the writer.
|
void |
setTarget(String path)
Sets the data sink for the writer.
|
void |
setWriteBuffer(int size)
Sets the size of the I/O buffer, in bytes,
to use for writes.
|
void |
setWriteOnClient(boolean enabled)
Set whether the writer should write a file on the client.
|
void |
setWriteSingleSink(boolean enabled)
Set whether the writer should produce a single output
file or multiple ones.
|
disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
disableParallelism, getInputPorts, getOutputPorts
protected final RecordPort input
protected final FormattingOptions options
protected AbstractWriter()
setTarget(ByteSink)
protected AbstractWriter(boolean includeDoneSignal)
includeDoneSignal
- indicates whether a
done signal port should be createdsetTarget(ByteSink)
protected AbstractWriter(String path, WriteMode mode)
If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
path
- the path to which to writemode
- how to handle existing filesprotected AbstractWriter(Path path, WriteMode mode)
If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
path
- the path to which to writemode
- how to handle existing filesprotected AbstractWriter(ByteSink target, WriteMode mode)
The writer can only be parallelized if the sink is fragmentable. In this case, each partition will be written as an independent sink. Otherwise, the writer will run non-parallel.
target
- the sink to which to writemode
- how to handle an existing sinkpublic RecordPort getInput()
getInput
in interface RecordSinkOperator
getInput
in interface SinkOperator<RecordPort>
public ByteSink getTarget()
public void setTarget(String path)
If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
path
- the path to which to writepublic void setTarget(Path path)
If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
path
- the path to which to writepublic void setTarget(ByteSink sink)
sink
- the sink to which to writepublic WriteMode getMode()
public void setMode(WriteMode mode)
mode
- the behavior to use for existing
filespublic FormattingOptions getFormatOptions()
public void setFormatOptions(FormattingOptions options)
options
- the format options to useDataFormatter
public int getWriteBuffer()
public void setWriteBuffer(int size)
size
- the size of the write bufferpublic boolean getWriteSingleSink()
true
if the writer should write
a single output, false
otherwise.public void setWriteSingleSink(boolean enabled)
enabled
- indicates whether a single output file
should be writtenpublic boolean getWriteOnClient()
true
if the writer should a file on the client, false
otherwise.public void setWriteOnClient(boolean enabled)
enabled
- indicates whether the writer should write a file on the client.public boolean isIgnoreSortOrder()
public void setIgnoreSortOrder(boolean ignoreSortOrder)
ignoreSortOrder
- whether to ignore sort order.public boolean getSaveMetadata()
public void setSaveMetadata(boolean saveMetadata)
saveMetadata
- whether metadata is savedprotected abstract DataFormat computeFormat(CompositionContext ctx)
WriteSink
operator. If an
implementation supports schema discovery, it must be
performed in this method.ctx
- the composition context for the current invocation
of compose(CompositionContext)
protected final void compose(CompositionContext ctx)
compose
in class CompositeOperator
ctx
- the contextCopyright © 2021 Actian Corporation. All rights reserved.