Module datarush.commons
Package com.pervasive.datarush.io

Class SplitOptions

java.lang.Object
com.pervasive.datarush.io.SplitOptions
public class SplitOptions extends Object
Settings which control the generation of splits on files.

  • Constructor Details

    • SplitOptions

      public SplitOptions()
      Create a collection with default settings.

  • Method Details

    • getNoPartialSplits

      public boolean getNoPartialSplits()
      Indicates whether breaking files into multiple splits is allowed.
      Returns:
      false if files can be broken into multiple splits, true if they cannot.

    • setNoPartialSplits

      public void setNoPartialSplits(boolean enabled)
      Controls whether single files are broken into multiple splits. If this enabled, splits are not allowed to select only a portion of a file. Some formats and use cases may not support sub-file granularity for parallelism, but can support parallelism at the per-file level. This flag supports this situation.

      By default, this behavior is disabled.

      Split generators must honor this setting.

      Parameters:
      enabled - indicates whether breaking files into multiple splits is disallowed

    • getTargetSize

      public int getTargetSize()
      Gets the target size for split generation.
      Returns:
      the size to use, in bytes, for splits

    • setTargetSize

      public void setTargetSize(int size)
      Sets the size to use, in bytes, for generating file splits. The file system should attempt to produce splits of this size, if possible. Modifying this size can increase (or decrease) the amount of parallelism when reading a file.

      If set to 0, the default size for the file system will be used.

      Parameters:
      size - the size, in bytes, to make file splits

    • set

      public void set(SplitOptions options)
      Copies the settings from another collection.
      Parameters:
      options - the settings to copy

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object