Module datarush.library
Package com.pervasive.datarush.operators.io

Class GlobbingByteSource

java.lang.Object
com.pervasive.datarush.operators.io.GlobbingByteSource
All Implemented Interfaces:
InputStreamSupplier, ByteSource
public class GlobbingByteSource extends Object implements ByteSource
A data source representing the concatenation of all files matching a pattern. Patterns are specified using a file globbing pattern as supported by FileClient.matchPaths(String). Directories which match the pattern are replaced by all files contained within the directory.

  • Constructor Details

    • GlobbingByteSource

      public GlobbingByteSource(String pattern)
      Defines a source consisting of all files matched by the specified pattern. The source produces the bytes resulting from a concatenation of the matched files. The order of the files in result is undefined.

      If any selected file is unreadable, an error will be raised when the source is used.

      Parameters:
      pattern - the file to use as a source

    • GlobbingByteSource

      public GlobbingByteSource(String pattern, UnreadableSourceAction onUnreadable)
      Defines a source consisting of all files matched by the specified pattern. The source produces the bytes resulting from a concatenation of the matched files. The order of the files in result is undefined.

      If any selected file is unreadable, it will be handled as specified.

      Parameters:
      pattern - the file to use as a source
      onUnreadable - specifies the action to take for unreadable files

  • Method Details

    • getPattern

      public String getPattern()
      Gets the globbing pattern used to match files.
      Returns:
      the file pattern for the source

    • getUnreadableAction

      public UnreadableSourceAction getUnreadableAction()
      Gets the configured behavior for handling unreadable source files.
      Returns:
      the configured behavior

    • authorize

      public ByteSource authorize(FileClient client)
      Description copied from interface: ByteSource
      Creates a new source with the same properties, but using the specified authorization.

      If a source is supposed to be used with a specific authorization context, this method should be called to produce a new source to use.

      Specified by:
      authorize in interface ByteSource
      Parameters:
      client - the authorization context to use for access
      Returns:
      a source using the provided authorization context

    • open

      public InputStream open() throws IOException
      Description copied from interface: ByteSource
      Opens the source for reading. The caller is responsible for closing the returned InputStream.
      Specified by:
      open in interface ByteSource
      Specified by:
      open in interface InputStreamSupplier
      Returns:
      a reader of the bytes from the source
      Throws:
      IOException - if an I/O error occurs while opening the source

    • generateSplits

      public SplitIterator generateSplits(SplitOptions options) throws IOException
      Description copied from interface: ByteSource
      Gets an iterator producing a set of DataSplit objects covering the source. The source is split as requested in the specified options, within the source's ability to meet the requirements.
      Specified by:
      generateSplits in interface ByteSource
      Parameters:
      options - configurable options to use in generating the splits
      Returns:
      an iterator over valid splits of the source
      Throws:
      IOException - if an I/O error occurs while generating splits

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • validate

      public ByteSource validate() throws IOException
      Description copied from interface: ByteSource
      Performs validation of the source configuration. This checks things such as the existence and accessibility of the source. It may also optionally rewrite the source to an equivalent one, doing file glob and directory expansion.
      Specified by:
      validate in interface ByteSource
      Returns:
      a valid source equivalent to this one
      Throws:
      IOException - if an I/O error occurs while validating the source