Module datarush.library
Package com.pervasive.datarush.operators.io.textfile

Class WriteARFF

java.lang.Object
com.pervasive.datarush.operators.AbstractLogicalOperator
com.pervasive.datarush.operators.CompositeOperator
com.pervasive.datarush.operators.io.AbstractWriter
com.pervasive.datarush.operators.io.textfile.AbstractTextWriter
com.pervasive.datarush.operators.io.textfile.WriteARFF
All Implemented Interfaces:
LogicalOperator, RecordSinkOperator, SinkOperator<RecordPort>
public class WriteARFF extends AbstractTextWriter
Write files using the Attribute-Relation File Format (ARFF). ARFF supports both sparse and dense formats. This can be specified by using the setFormatMode(ARFFMode) method. The default mode is sparse. When the input data is highly sparse, the sparse mode can save space since it only writes non-zero values. However, if the data is not very sparse, using sparse mode can actually cause the resultant file to be larger; use dense mode for dense data.

Note that when writing sparse data that contains enumerated types, the zero ordinal category will always be treated as sparse and so will not appear in the output file. This is normal and expected behavior. The enumerated type is captured in the meta-data. When the file is read, the zero ordinal values will be restored as expected.

  • Constructor Details

    • WriteARFF

      public WriteARFF()
      Writes ARFF data to an empty target with default settings. The target must be set before execution or an error will be raised.
      See Also:

    • WriteARFF

      public WriteARFF(String path, WriteMode mode)
      Writes ARFF data to the specified path in the given mode, using default settings.

      If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.

      Parameters:
      path - the path to which to write
      mode - how to handle existing files

    • WriteARFF

      public WriteARFF(Path path, WriteMode mode)
      Writes ARFF data to the specified path in the given mode, using default settings.

      If the writer is parallelized, this is interpreted as a directory in which each partition will write a fragment of the entire input stream. Otherwise, it is interpreted as the file to write.

      Parameters:
      path - the path to which to write
      mode - how to handle existing files

    • WriteARFF

      public WriteARFF(ByteSink target, WriteMode mode)
      Writes ARFF data to the specified target sink in the given mode.

      The writer can only be parallelized if the sink is fragmentable. In this case, each partition will be written as an independent sink. Otherwise, the writer will run non-parallel.

      Parameters:
      target - the sink to which to write
      mode - how to handle an existing sink

  • Method Details

    • getRelationName

      public String getRelationName()
      Get the value of the relation name property.
      Returns:
      relation name

    • setRelationName

      public void setRelationName(String relationName)
      Set the relation name attribute. In ARFF, the relation name is captured in the meta-data with the tag @attribute.
      Parameters:
      relationName - name of the relation

    • addComment

      public void addComment(String comment)
      Add a comment line that will be written to the ARFF meta-data section of the file. Multiple comments may be added. They will be written in the order provided.
      Parameters:
      comment - a line of commentary to add to the output file

    • setComments

      public void setComments(List<String> comments)
      Set the comments lines to write to the ARFF meta-data section of the file.
      Parameters:
      comments - lines of commentary to add to the output file

    • getComments

      public List<String> getComments()
      Gets the comments lines current set to be written.
      Returns:
      the lines of commentary to add to the output file

    • setRecordSeparator

      public void setRecordSeparator(String separator)
      Set the record separator. This string will be output at the end of each record written. The default value is the system dependent record separator.
      Parameters:
      separator - text separating records

    • getRecordSeparator

      public String getRecordSeparator()
      Get the record separator.
      Returns:
      record separator

    • getFormatMode

      public ARFFMode getFormatMode()
      Get the value of the mode property.
      Returns:
      write mode

    • setFormatMode

      public void setFormatMode(ARFFMode mode)
      Set the mode with which to write the file. The supported modes are SPARSE and DENSE.
      Parameters:
      mode - write mode; either sparse or dense

    • getSchema

      public TextRecord getSchema()
      Get the configured schema for this reader instance.
      Returns:
      configured schema (may be null)

    • setFieldDelimiter

      public void setFieldDelimiter(char delimiter)
      Set the field delimiter to use when writing the file contents. A single quote is used by default. The only supported values are a single quote and a double quote.
      Parameters:
      delimiter - the delimiter to used for field values containing spaces

    • getFieldDelimiter

      public char getFieldDelimiter()
      Get the configured field delimiter property value.
      Returns:
      configured field delimiter

    • setSchema

      public void setSchema(TextRecord schema)
      Set the schema to use for the provided data. A schema is not required. If a schema is not provided, a default schema will be constructed using default formatting options for all fields. A schema is useful to explicitly specify the formatting options for fields. This is especially useful for date and timestamp types, but is applicable to all types.
      Parameters:
      schema - definition of field ordering, names, types and formats

    • computeFormat

      protected DataFormat computeFormat(CompositionContext ctx)
      Description copied from class: AbstractWriter
      Determines the data format for the target. The returned format is used during composition to construct a WriteSink operator. If an implementation supports schema discovery, it must be performed in this method.
      Specified by:
      computeFormat in class AbstractWriter
      Parameters:
      ctx - the composition context for the current invocation of AbstractWriter.compose(CompositionContext)
      Returns:
      the target format to use