Module datarush.library
Package com.pervasive.datarush.operators.io

Class ConcatenatedByteSource

java.lang.Object
com.pervasive.datarush.operators.io.ConcatenatedByteSource
All Implemented Interfaces:
InputStreamSupplier, ByteSource
public class ConcatenatedByteSource extends Object implements ByteSource
A data source which concatenates multiple sources. The resulting source provides the bytes of all sources, in the order specified when created.

  • Constructor Details

    • ConcatenatedByteSource

      public ConcatenatedByteSource(List<? extends ByteSource> sources)
      Creates a data source representing the concatenation of the specified sources.
      Parameters:
      sources - the data sources to concatenate

  • Method Details

    • getSources

      public List<ByteSource> getSources()
      Gets the sources which comprise the combined source.
      Returns:
      the concatenated sources

    • authorize

      public ByteSource authorize(FileClient client)
      Description copied from interface: ByteSource
      Creates a new source with the same properties, but using the specified authorization.

      If a source is supposed to be used with a specific authorization context, this method should be called to produce a new source to use.

      Specified by:
      authorize in interface ByteSource
      Parameters:
      client - the authorization context to use for access
      Returns:
      a source using the provided authorization context

    • open

      public InputStream open() throws IOException
      Description copied from interface: ByteSource
      Opens the source for reading. The caller is responsible for closing the returned InputStream.
      Specified by:
      open in interface ByteSource
      Specified by:
      open in interface InputStreamSupplier
      Returns:
      a reader of the bytes from the source
      Throws:
      IOException - if an I/O error occurs while opening the source

    • generateSplits

      public SplitIterator generateSplits(SplitOptions options) throws IOException
      Description copied from interface: ByteSource
      Gets an iterator producing a set of DataSplit objects covering the source. The source is split as requested in the specified options, within the source's ability to meet the requirements.
      Specified by:
      generateSplits in interface ByteSource
      Parameters:
      options - configurable options to use in generating the splits
      Returns:
      an iterator over valid splits of the source
      Throws:
      IOException - if an I/O error occurs while generating splits

    • validate

      public ByteSource validate() throws IOException
      Description copied from interface: ByteSource
      Performs validation of the source configuration. This checks things such as the existence and accessibility of the source. It may also optionally rewrite the source to an equivalent one, doing file glob and directory expansion.
      Specified by:
      validate in interface ByteSource
      Returns:
      a valid source equivalent to this one
      Throws:
      IOException - if an I/O error occurs while validating the source

    • getConcatenatedSource

      public static ByteSource getConcatenatedSource(List<Path> paths)
      Creates a source which is the concatenation of the specified paths, in order. If a path identifies a directory, it is interpreted as all files in the directory. If any selected file is unreadable, an error will be raised when the source is used.
      Parameters:
      paths - the paths to concatenate
      Returns:
      a source concatenating the identified files
      See Also:

    • getConcatenatedSource

      public static ByteSource getConcatenatedSource(List<Path> paths, UnreadableSourceAction onUnreadable)
      Creates a source which is the concatenation of the specified paths, in order. If a path identifies a directory, it is interpreted as all files in the directory. If any selected file is unreadable, it will be handled as specified.
      Parameters:
      paths - the paths to concatenate
      onUnreadable - specifies the action to take for unreadable files
      Returns:
      a source concatenating the identified files
      See Also: