Module datarush.library
Package com.pervasive.datarush.operators.io.textfile

Class ARFFDataFormat

java.lang.Object
com.pervasive.datarush.operators.io.textfile.ARFFDataFormat
All Implemented Interfaces:
DataFormat
public class ARFFDataFormat extends Object implements DataFormat
Describes an instance of ARFF data. Normally, it is not necessary construct these directly. Instead, use ReadARFF and WriteARFF to access data stored as ARRF.

  • Constructor Details

    • ARFFDataFormat

      public ARFFDataFormat(ARFFMode mode, TextRecord schema, CharsetEncoding encoding, String recordSeparator, char fieldDelimiter, String relationName, List<String> comments)
      Create an ARFF data format for writing to an ARFF file.
      Parameters:
      mode - indicates the mode for representing record data
      schema - the schema to use for records. This provides field names (and positions) as well as the formatting of field values.
      encoding - character set definition for data encoding
      recordSeparator - characters to use as a separator between records
      fieldDelimiter - the character use to delimit field values when the value contains spaces
      relationName - name of the relation
      comments - comments to associate with data written using this format.

    • ARFFDataFormat

      public ARFFDataFormat(ARFFMode mode, TextRecord schema, CharsetEncoding encoding, char fieldDelimiter)
      Create an ARFF data format for reading an ARFF file.
      Parameters:
      mode - indicates the mode for representing record data
      schema - the schema to use for records. This provides field names (and positions) as well as the formatting of field values. This schema also controls the output type of the reader.
      encoding - character set definition for data encoding
      fieldDelimiter - the character use to delimit field values when the value contains spaces

  • Method Details

    • getType

      public RecordTokenType getType()
      Description copied from interface: DataFormat
      Gets the record type associated with the format. Records produced by the associated parser or consumed by the associated formatter will be of this type.

      For many formats, this may be derived from a schema object describing the format layout.

      Specified by:
      getType in interface DataFormat
      Returns:
      the format's record type

    • getMetadata

      public FileMetadata getMetadata()
      Description copied from interface: DataFormat
      Gets the metadata associated with the format. Records produces by the associated parser or consumed by the associated formatter will use this metadata.
      Specified by:
      getMetadata in interface DataFormat
      Returns:
      the format's metadata

    • setMetadata

      public void setMetadata(FileMetadata metadata)
      Description copied from interface: DataFormat
      Sets the metadata associated with the format.
      Specified by:
      setMetadata in interface DataFormat

    • readMetadata

      public FileMetadata readMetadata(FileClient fileClient, ByteSource source)
      Description copied from interface: DataFormat
      Reads the metadata associated with the format.
      Specified by:
      readMetadata in interface DataFormat
      Parameters:
      fileClient - client used to read file
      source - location of the files

    • writeMetadata

      public void writeMetadata(FileMetadata metadata, FileClient fileClient, ByteSink target)
      Description copied from interface: DataFormat
      Writes the provided metadata associated with the format.
      Specified by:
      writeMetadata in interface DataFormat
      Parameters:
      metadata - the metadata to write
      fileClient - client used to write file

    • createParser

      public DataFormat.DataParser createParser(ParsingOptions options)
      Description copied from interface: DataFormat
      Create a new parser for the format using the specified parsing options.
      Specified by:
      createParser in interface DataFormat
      Parameters:
      options - parsing options to use
      Returns:
      a new parser for reading external data

    • createWriter

      public DataFormat.DataFormatter createWriter(FormattingOptions options)
      Description copied from interface: DataFormat
      Create a new writer for the format using the specified formatting options.
      Specified by:
      createWriter in interface DataFormat
      Parameters:
      options - formatting options to use
      Returns:
      a new formatter for writing external data

    • isSplittable

      public boolean isSplittable()
      Indicates if the format supports parsing of subsections of a file.

      A format should only return true if it can, at least in some situations, support this sort of parsing. If a format requires reading the entire file, it must return false.

      If a format is not splittable, a file in the format cannot be parsed in parallel; however, individual files can still be parsed independently in parallel, as when reading the contents of a directory or using a file globbing pattern.

      ARFF is a splittable format.

      Specified by:
      isSplittable in interface DataFormat
      Returns:
      true if the format supports parsing only a portion of the file, false otherwise