com.pervasive.datarush.operators.io

Class BasicByteSource

java.lang.Object

com.pervasive.datarush.operators.io.BasicByteSource

All Implemented Interfaces:

InputStreamSupplier, ByteSource

public class BasicByteSource
extends Object
implements ByteSource

A data source identified by a Path. Typically, these represent readable files.

Constructor Summary

Constructors

Constructor and Description
BasicByteSource(Path path) Creates a data source for the named path.
BasicByteSource(Path path, CompressionFormat format) Creates a data source for the named path.
BasicByteSource(Path path, UnreadableSourceAction onUnreadable) Creates a data source for the named path.
BasicByteSource(String path) Creates a data source for the named path.

Method Summary

Modifier and Type	Method and Description
ByteSource	authorize(FileClient client) Creates a new source with the same properties, but using the specified authorization.
SplitIterator	generateSplits(SplitOptions options) Gets an iterator producing a set of DataSplit objects covering the source.
CompressionFormat	getCompression() Gets the compression format expected for reading.
Path	getPath() Gets the path identifying the byte source.
UnreadableSourceAction	getUnreadableAction() Gets the configured behavior for handling unreadable source files.
InputStream	open() Opens the source for reading.
String	toString()
ByteSource	validate() Performs validation of the source configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

BasicByteSource

public BasicByteSource(String path)

Creates a data source for the named path. If the path identifies a directory, it is interpreted to mean all files in the directory.

The last element of the path is used to automatically determine if the source is compressed, and if so, what the format is. If this is not desired, use BasicByteSource(Path, CompressionFormat) instead.

Parameters:

path - the file to use as a source

BasicByteSource

public BasicByteSource(Path path)

Creates a data source for the named path. If the path identifies a directory, it is interpreted to mean all files in the directory. If any selected file is unreadable, an error will be raised when the source is used.

The last element of the path is used to automatically determine if the source is compressed, and if so, what the format is. If this is not desired, use BasicByteSource(Path, CompressionFormat) instead.

Parameters:

path - the file to use as a source

BasicByteSource

public BasicByteSource(Path path,
                       UnreadableSourceAction onUnreadable)

Creates a data source for the named path. If the path identifies a directory, it is interpreted to mean all files in the directory. If any selected file is unreadable, it will be handled as specified.

The last element of the path is used to automatically determine if the source is compressed, and if so, what the format is. If this is not desired, use BasicByteSource(Path, CompressionFormat) instead.

Parameters:

path - the file to use as a source

onUnreadable - specifies the action to take for unreadable files

BasicByteSource

public BasicByteSource(Path path,
                       CompressionFormat format)

Creates a data source for the named path. Data will be read assuming the specified compression format.

If the path identifies a directory, it is interpreted to mean all files in the directory. If any selected file is unreadable, an error will be raised when the source is used.

Parameters:

path - the file/directory to read

format - the compression format of the identified file(s)

Method Detail

getPath

public Path getPath()

Gets the path identifying the byte source.

Returns:

the path

getCompression

public CompressionFormat getCompression()

Gets the compression format expected for reading.

Returns:

the expected source compression format

getUnreadableAction

public UnreadableSourceAction getUnreadableAction()

Gets the configured behavior for handling unreadable source files.

Returns:

the configured behavior

authorize

public ByteSource authorize(FileClient client)

Description copied from interface: ByteSource

Creates a new source with the same properties, but using the specified authorization.

If a source is supposed to be used with a specific authorization context, this method should be called to produce a new source to use.

Specified by:

authorize in interface ByteSource

Parameters:

client - the authorization context to use for access

Returns:

a source using the provided authorization context

open

public InputStream open()
                 throws IOException

Description copied from interface: ByteSource

Opens the source for reading. The caller is responsible for closing the returned InputStream.

Specified by:

open in interface InputStreamSupplier

Specified by:

open in interface ByteSource

Returns:

a reader of the bytes from the source

Throws:

IOException - if an I/O error occurs while opening the source

generateSplits

public SplitIterator generateSplits(SplitOptions options)
                             throws IOException

Description copied from interface: ByteSource

Gets an iterator producing a set of DataSplit objects covering the source. The source is split as requested in the specified options, within the source's ability to meet the requirements.

Specified by:

generateSplits in interface ByteSource

Parameters:

options - configurable options to use in generating the splits

Returns:

an iterator over valid splits of the source

Throws:

IOException - if an I/O error occurs while generating splits

toString

public String toString()

Overrides:

toString in class Object

validate

public ByteSource validate()
                    throws IOException

Description copied from interface: ByteSource

Performs validation of the source configuration. This checks things such as the existence and accessibility of the source. It may also optionally rewrite the source to an equivalent one, doing file glob and directory expansion.

Specified by:

validate in interface ByteSource

Returns:

a valid source equivalent to this one

Throws:

IOException - if an I/O error occurs while validating the source