Module datarush.library
Package com.pervasive.datarush.operators.io

Class ConcatenatedByteSource

  • All Implemented Interfaces:
    InputStreamSupplier, ByteSource
    public class ConcatenatedByteSource
extends Object
implements ByteSource
    A data source which concatenates multiple sources. The resulting source provides the bytes of all sources, in the order specified when created.

    • Constructor Detail

      • ConcatenatedByteSource

        public ConcatenatedByteSource​(List<? extends ByteSource> sources)
        Creates a data source representing the concatenation of the specified sources.
        Parameters:
        sources - the data sources to concatenate

    • Method Detail

      • getSources

        public List<ByteSource> getSources()
        Gets the sources which comprise the combined source.
        Returns:
        the concatenated sources

      • authorize

        public ByteSource authorize​(FileClient client)
        Description copied from interface: ByteSource
        Creates a new source with the same properties, but using the specified authorization.

        If a source is supposed to be used with a specific authorization context, this method should be called to produce a new source to use.

        Specified by:
        authorize in interface ByteSource
        Parameters:
        client - the authorization context to use for access
        Returns:
        a source using the provided authorization context

      • open

        public InputStream open()
                 throws IOException
        Description copied from interface: ByteSource
        Opens the source for reading. The caller is responsible for closing the returned InputStream.
        Specified by:
        open in interface ByteSource
        Specified by:
        open in interface InputStreamSupplier
        Returns:
        a reader of the bytes from the source
        Throws:
        IOException - if an I/O error occurs while opening the source

      • generateSplits

        public SplitIterator generateSplits​(SplitOptions options)
                             throws IOException
        Description copied from interface: ByteSource
        Gets an iterator producing a set of DataSplit objects covering the source. The source is split as requested in the specified options, within the source's ability to meet the requirements.
        Specified by:
        generateSplits in interface ByteSource
        Parameters:
        options - configurable options to use in generating the splits
        Returns:
        an iterator over valid splits of the source
        Throws:
        IOException - if an I/O error occurs while generating splits

      • validate

        public ByteSource validate()
                    throws IOException
        Description copied from interface: ByteSource
        Performs validation of the source configuration. This checks things such as the existence and accessibility of the source. It may also optionally rewrite the source to an equivalent one, doing file glob and directory expansion.
        Specified by:
        validate in interface ByteSource
        Returns:
        a valid source equivalent to this one
        Throws:
        IOException - if an I/O error occurs while validating the source

      • getConcatenatedSource

        public static ByteSource getConcatenatedSource​(List<Path> paths)
        Creates a source which is the concatenation of the specified paths, in order. If a path identifies a directory, it is interpreted as all files in the directory. If any selected file is unreadable, an error will be raised when the source is used.
        Parameters:
        paths - the paths to concatenate
        Returns:
        a source concatenating the identified files
        See Also:
        BasicByteSource

      • getConcatenatedSource

        public static ByteSource getConcatenatedSource​(List<Path> paths,
                                               UnreadableSourceAction onUnreadable)
        Creates a source which is the concatenation of the specified paths, in order. If a path identifies a directory, it is interpreted as all files in the directory. If any selected file is unreadable, it will be handled as specified.
        Parameters:
        paths - the paths to concatenate
        onUnreadable - specifies the action to take for unreadable files
        Returns:
        a source concatenating the identified files
        See Also:
        BasicByteSource