com.pervasive.datarush.operators.io

Class ConcatenatedByteSource

java.lang.Object

com.pervasive.datarush.operators.io.ConcatenatedByteSource

All Implemented Interfaces:

InputStreamSupplier, ByteSource

public class ConcatenatedByteSource
extends Object
implements ByteSource

A data source which concatenates multiple sources. The resulting source provides the bytes of all sources, in the order specified when created.

Constructor Summary

Constructors

Constructor and Description
ConcatenatedByteSource(List<? extends ByteSource> sources) Creates a data source representing the concatenation of the specified sources.

Method Summary

Modifier and Type	Method and Description
ByteSource	authorize(FileClient client) Creates a new source with the same properties, but using the specified authorization.
SplitIterator	generateSplits(SplitOptions options) Gets an iterator producing a set of DataSplit objects covering the source.
static ByteSource	getConcatenatedSource(List<Path> paths) Creates a source which is the concatenation of the specified paths, in order.
static ByteSource	getConcatenatedSource(List<Path> paths, UnreadableSourceAction onUnreadable) Creates a source which is the concatenation of the specified paths, in order.
List<ByteSource>	getSources() Gets the sources which comprise the combined source.
InputStream	open() Opens the source for reading.
ByteSource	validate() Performs validation of the source configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ConcatenatedByteSource

public ConcatenatedByteSource(List<? extends ByteSource> sources)

Creates a data source representing the concatenation of the specified sources.

Parameters:

sources - the data sources to concatenate

Method Detail

getSources

public List<ByteSource> getSources()

Gets the sources which comprise the combined source.

Returns:

the concatenated sources

authorize

public ByteSource authorize(FileClient client)

Description copied from interface: ByteSource

Creates a new source with the same properties, but using the specified authorization.

If a source is supposed to be used with a specific authorization context, this method should be called to produce a new source to use.

Specified by:

authorize in interface ByteSource

Parameters:

client - the authorization context to use for access

Returns:

a source using the provided authorization context

open

public InputStream open()
                 throws IOException

Description copied from interface: ByteSource

Opens the source for reading. The caller is responsible for closing the returned InputStream.

Specified by:

open in interface InputStreamSupplier

Specified by:

open in interface ByteSource

Returns:

a reader of the bytes from the source

Throws:

IOException - if an I/O error occurs while opening the source

generateSplits

public SplitIterator generateSplits(SplitOptions options)
                             throws IOException

Description copied from interface: ByteSource

Gets an iterator producing a set of DataSplit objects covering the source. The source is split as requested in the specified options, within the source's ability to meet the requirements.

Specified by:

generateSplits in interface ByteSource

Parameters:

options - configurable options to use in generating the splits

Returns:

an iterator over valid splits of the source

Throws:

IOException - if an I/O error occurs while generating splits

validate

public ByteSource validate()
                    throws IOException

Description copied from interface: ByteSource

Performs validation of the source configuration. This checks things such as the existence and accessibility of the source. It may also optionally rewrite the source to an equivalent one, doing file glob and directory expansion.

Specified by:

validate in interface ByteSource

Returns:

a valid source equivalent to this one

Throws:

IOException - if an I/O error occurs while validating the source

getConcatenatedSource

public static ByteSource getConcatenatedSource(List<Path> paths)

Creates a source which is the concatenation of the specified paths, in order. If a path identifies a directory, it is interpreted as all files in the directory. If any selected file is unreadable, an error will be raised when the source is used.

Parameters:

paths - the paths to concatenate

Returns:

a source concatenating the identified files

See Also:

BasicByteSource

getConcatenatedSource

public static ByteSource getConcatenatedSource(List<Path> paths,
                                               UnreadableSourceAction onUnreadable)

Creates a source which is the concatenation of the specified paths, in order. If a path identifies a directory, it is interpreted as all files in the directory. If any selected file is unreadable, it will be handled as specified.

Parameters:

paths - the paths to concatenate

onUnreadable - specifies the action to take for unreadable files

Returns:

a source concatenating the identified files

See Also:

BasicByteSource