AbstractWriter (Dataflow Library Distribution Project 6.8.0-1 API)

java.lang.Object
- com.pervasive.datarush.operators.AbstractLogicalOperator
- - com.pervasive.datarush.operators.CompositeOperator
  - - com.pervasive.datarush.operators.io.AbstractWriter

All Implemented Interfaces:

LogicalOperator, RecordSinkOperator, SinkOperator<RecordPort>

Direct Known Subclasses:

AbstractTextWriter, WriteAvro, WriteORC, WriteStagingDataset
```
public abstract class AbstractWriter
extends CompositeOperator
implements RecordSinkOperator
```
A generic writer of byte data representing a stream of records. The writer encompasses the basic attributes any such reader should have:
- a sink for bytes, as identified by a ByteSink. This is most often a file, so convenience methods for specifying the target as a file are provided.
- common I/O tunables, such as buffer sizes
An AbstractWriter wraps a WriteSink operator. Implementations provide an interface specific to an appropriate class of files - delimited text files, as an example - hiding the more complex model used to describe data parsing in general. The composition structure is the same for all readers with only DataFormat used differing between implementations.

Field Summary

Fields
Modifier and Type	Field and Description
`protected RecordPort`	`input` The input port of the read operator
`protected FormattingOptions`	`options` Container for options related to formatting

Constructor Summary

Constructors
Modifier	Constructor and Description
`protected`	`AbstractWriter()` Writes an empty target with default settings.
`protected`	`AbstractWriter(boolean includeDoneSignal)` Writes an empty target with default settings, optionally providing a port for signaling completion of the write.
`protected`	`AbstractWriter(ByteSink target, WriteMode mode)` Writes to the specified target sink in the given mode.
`protected`	`AbstractWriter(Path path, WriteMode mode)` Writes to the specified path in the given mode.
`protected`	`AbstractWriter(String path, WriteMode mode)` Writes to the specified path in the given mode.

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`protected void`	`compose(CompositionContext ctx)` Composes a writer for the target, using configured options and a derived format.
`protected abstract DataFormat`	`computeFormat(CompositionContext ctx)` Determines the data format for the target.
`FormattingOptions`	`getFormatOptions()` Gets the formatting options used by the writer.
`RecordPort`	`getInput()` Gets the record port providing the records to write to the target sink.
`WriteMode`	`getMode()` Gets how existing files should be handled by the writer.
`boolean`	`getSaveMetadata()` If set to true, the writer will attempt to save the metadata with the data if the implementation supports it.
`ByteSink`	`getTarget()` Gets the data sink for the writer.
`int`	`getWriteBuffer()` Gets the size of the I/O buffer, in bytes, to use for writes.
`boolean`	`getWriteOnClient()` Indicates whether the writer should write a file on the client.
`boolean`	`getWriteSingleSink()` Indicates whether the writer produces a single output file.
`boolean`	`isIgnoreSortOrder()` If set to true, the writer will write in any order.
`void`	`setFormatOptions(FormattingOptions options)` Sets the formatting options used by the reader.
`void`	`setIgnoreSortOrder(boolean ignoreSortOrder)` If set to true, the writer will write in any order.
`void`	`setMode(WriteMode mode)` Sets how the writer should handle an existing target.
`void`	`setSaveMetadata(boolean saveMetadata)` If set to true, the writer will attempt to save the metadata with the data if the implementation supports it.
`void`	`setTarget(ByteSink sink)` Sets the data sink for the writer.
`void`	`setTarget(Path path)` Sets the data sink for the writer.
`void`	`setTarget(String path)` Sets the data sink for the writer.
`void`	`setWriteBuffer(int size)` Sets the size of the I/O buffer, in bytes, to use for writes.
`void`	`setWriteOnClient(boolean enabled)` Set whether the writer should write a file on the client.
`void`	`setWriteSingleSink(boolean enabled)` Set whether the writer should produce a single output file or multiple ones.

Methods inherited from class com.pervasive.datarush.operators.AbstractLogicalOperator
disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.pervasive.datarush.operators.LogicalOperator
disableParallelism, getInputPorts, getOutputPorts

- Field Detail
  - input
```
protected final RecordPort input
```
    The input port of the read operator
  - options
```
protected final FormattingOptions options
```
    Container for options related to formatting
- Constructor Detail
  - AbstractWriter
```
protected AbstractWriter()
```
    Writes an empty target with default settings. The target must be set before execution or an error will be raised.
    
    See Also:
    
    setTarget(ByteSink)
  - AbstractWriter
```
protected AbstractWriter(boolean includeDoneSignal)
```
    Writes an empty target with default settings, optionally providing a port for signaling completion of the write. The target must be set before execution or an error will be raised.
    
    Parameters:
    
    includeDoneSignal - indicates whether a done signal port should be created
    
    See Also:
    
    setTarget(ByteSink)
  - AbstractWriter
```
protected AbstractWriter(String path,
                         WriteMode mode)
```
    Writes to the specified path in the given mode.
    If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
    
    Parameters:
    
    path - the path to which to write
    
    mode - how to handle existing files
  - AbstractWriter
```
protected AbstractWriter(Path path,
                         WriteMode mode)
```
    Writes to the specified path in the given mode.
    If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
    
    Parameters:
    
    path - the path to which to write
    
    mode - how to handle existing files
  - AbstractWriter
```
protected AbstractWriter(ByteSink target,
                         WriteMode mode)
```
    Writes to the specified target sink in the given mode.
    The writer can only be parallelized if the sink is fragmentable. In this case, each partition will be written as an independent sink. Otherwise, the writer will run non-parallel.
    
    Parameters:
    
    target - the sink to which to write
    
    mode - how to handle an existing sink
- Method Detail
  - getInput
```
public RecordPort getInput()
```
    Gets the record port providing the records to write to the target sink.
    
    Specified by:
    
    getInput in interface RecordSinkOperator
    
    Specified by:
    
    getInput in interface SinkOperator<RecordPort>
    
    Returns:
    
    the input port for records to write
  - getTarget
```
public ByteSink getTarget()
```
    Gets the data sink for the writer.
    
    Returns:
    
    the write sink
  - setTarget
```
public void setTarget(String path)
```
    Sets the data sink for the writer.
    If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
    
    Parameters:
    
    path - the path to which to write
  - setTarget
```
public void setTarget(Path path)
```
    Sets the data sink for the writer.
    If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.
    
    Parameters:
    
    path - the path to which to write
  - setTarget
```
public void setTarget(ByteSink sink)
```
    Sets the data sink for the writer.
    
    Parameters:
    
    sink - the sink to which to write
  - getMode
```
public WriteMode getMode()
```
    Gets how existing files should be handled by the writer.
    
    Returns:
    
    the configured behavior with respect to target files
  - setMode
```
public void setMode(WriteMode mode)
```
    Sets how the writer should handle an existing target.
    
    Parameters:
    
    mode - the behavior to use for existing files
  - getFormatOptions
```
public FormattingOptions getFormatOptions()
```
    Gets the formatting options used by the writer.
    
    Returns:
    
    the format options
  - setFormatOptions
```
public void setFormatOptions(FormattingOptions options)
```
    Sets the formatting options used by the reader. This sets all format options at once.
    
    Parameters:
    
    options - the format options to use
    
    See Also:
    
    DataFormatter
  - getWriteBuffer
```
public int getWriteBuffer()
```
    Gets the size of the I/O buffer, in bytes, to use for writes.
    
    Returns:
    
    the size of the write buffer
  - setWriteBuffer
```
public void setWriteBuffer(int size)
```
    Sets the size of the I/O buffer, in bytes, to use for writes. The default size is 64K.
    
    Parameters:
    
    size - the size of the write buffer
  - getWriteSingleSink
```
public boolean getWriteSingleSink()
```
    Indicates whether the writer produces a single output file.
    
    Returns:
    
    true if the writer should write a single output, false otherwise.
  - setWriteSingleSink
```
public void setWriteSingleSink(boolean enabled)
```
    Set whether the writer should produce a single output file or multiple ones. By default, an output file will be produced for each partition, if the target sink supports this.
    
    Parameters:
    
    enabled - indicates whether a single output file should be written
  - getWriteOnClient
```
public boolean getWriteOnClient()
```
    Indicates whether the writer should write a file on the client. This also disables the parallelism on the sink.
    
    Returns:
    
    true if the writer should a file on the client, false otherwise.
  - setWriteOnClient
```
public void setWriteOnClient(boolean enabled)
```
    Set whether the writer should write a file on the client. This has the effect of disabling the parallelism on the sink.
    
    Parameters:
    
    enabled - indicates whether the writer should write a file on the client.
  - isIgnoreSortOrder
```
public boolean isIgnoreSortOrder()
```
    If set to true, the writer will write in any order. If false, the writer will preserve the current sorted state of the input.
    
    Returns:
    
    whether to ignore sort order.
  - setIgnoreSortOrder
```
public void setIgnoreSortOrder(boolean ignoreSortOrder)
```
    If set to true, the writer will write in any order. If false, the writer will preserve the current sorted state of the input.
    
    Parameters:
    
    ignoreSortOrder - whether to ignore sort order.
  - getSaveMetadata
```
public boolean getSaveMetadata()
```
    If set to true, the writer will attempt to save the metadata with the data if the implementation supports it. Otherwise metadata will not be saved.
    
    Returns:
    
    whether metadata is saved
  - setSaveMetadata
```
public void setSaveMetadata(boolean saveMetadata)
```
    If set to true, the writer will attempt to save the metadata with the data if the implementation supports it. Otherwise metadata will not be saved.
    
    Parameters:
    
    saveMetadata - whether metadata is saved
  - computeFormat
```
protected abstract DataFormat computeFormat(CompositionContext ctx)
```
    Determines the data format for the target. The returned format is used during composition to construct a WriteSink operator. If an implementation supports schema discovery, it must be performed in this method.
    
    Parameters:
    
    ctx - the composition context for the current invocation of compose(CompositionContext)
    
    Returns:
    
    the target format to use
  - compose
```
protected final void compose(CompositionContext ctx)
```
    Composes a writer for the target, using configured options and a derived format.
    
    Specified by:
    
    compose in class CompositeOperator
    
    Parameters:
    
    ctx - the context

Class AbstractWriter

Field Summary

Constructor Summary

Method Summary

Methods inherited from class com.pervasive.datarush.operators.AbstractLogicalOperator

Methods inherited from class java.lang.Object

Methods inherited from interface com.pervasive.datarush.operators.LogicalOperator

Field Detail

input

options

Constructor Detail

AbstractWriter

AbstractWriter

AbstractWriter

AbstractWriter

AbstractWriter

Method Detail

getInput

getTarget

setTarget

setTarget

setTarget

getMode

setMode

getFormatOptions

setFormatOptions

getWriteBuffer

setWriteBuffer

getWriteSingleSink

setWriteSingleSink

getWriteOnClient

setWriteOnClient

isIgnoreSortOrder

setIgnoreSortOrder

getSaveMetadata

setSaveMetadata

computeFormat

compose