com.pervasive.datarush.operators.io

Class GlobbingByteSource

java.lang.Object

com.pervasive.datarush.operators.io.GlobbingByteSource

All Implemented Interfaces:

InputStreamSupplier, ByteSource

public class GlobbingByteSource
extends Object
implements ByteSource

A data source representing the concatenation of all files matching a pattern. Patterns are specified using a file globbing pattern as supported by FileClient.matchPaths(String). Directories which match the pattern are replaced by all files contained within the directory.

Constructor Summary

Constructors

Constructor and Description
GlobbingByteSource(String pattern) Defines a source consisting of all files matched by the specified pattern.
GlobbingByteSource(String pattern, UnreadableSourceAction onUnreadable) Defines a source consisting of all files matched by the specified pattern.

Method Summary

Modifier and Type	Method and Description
ByteSource	authorize(FileClient client) Creates a new source with the same properties, but using the specified authorization.
SplitIterator	generateSplits(SplitOptions options) Gets an iterator producing a set of DataSplit objects covering the source.
String	getPattern() Gets the globbing pattern used to match files.
UnreadableSourceAction	getUnreadableAction() Gets the configured behavior for handling unreadable source files.
InputStream	open() Opens the source for reading.
String	toString()
ByteSource	validate() Performs validation of the source configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

GlobbingByteSource

public GlobbingByteSource(String pattern)

Defines a source consisting of all files matched by the specified pattern. The source produces the bytes resulting from a concatenation of the matched files. The order of the files in result is undefined.

If any selected file is unreadable, an error will be raised when the source is used.

Parameters:

pattern - the file to use as a source

GlobbingByteSource

public GlobbingByteSource(String pattern,
                          UnreadableSourceAction onUnreadable)

Defines a source consisting of all files matched by the specified pattern. The source produces the bytes resulting from a concatenation of the matched files. The order of the files in result is undefined.

If any selected file is unreadable, it will be handled as specified.

Parameters:

pattern - the file to use as a source

onUnreadable - specifies the action to take for unreadable files

Method Detail

getPattern

public String getPattern()

Gets the globbing pattern used to match files.

Returns:

the file pattern for the source

getUnreadableAction

public UnreadableSourceAction getUnreadableAction()

Gets the configured behavior for handling unreadable source files.

Returns:

the configured behavior

authorize

public ByteSource authorize(FileClient client)

Description copied from interface: ByteSource

Creates a new source with the same properties, but using the specified authorization.

If a source is supposed to be used with a specific authorization context, this method should be called to produce a new source to use.

Specified by:

authorize in interface ByteSource

Parameters:

client - the authorization context to use for access

Returns:

a source using the provided authorization context

open

public InputStream open()
                 throws IOException

Description copied from interface: ByteSource

Opens the source for reading. The caller is responsible for closing the returned InputStream.

Specified by:

open in interface InputStreamSupplier

Specified by:

open in interface ByteSource

Returns:

a reader of the bytes from the source

Throws:

IOException - if an I/O error occurs while opening the source

generateSplits

public SplitIterator generateSplits(SplitOptions options)
                             throws IOException

Description copied from interface: ByteSource

Gets an iterator producing a set of DataSplit objects covering the source. The source is split as requested in the specified options, within the source's ability to meet the requirements.

Specified by:

generateSplits in interface ByteSource

Parameters:

options - configurable options to use in generating the splits

Returns:

an iterator over valid splits of the source

Throws:

IOException - if an I/O error occurs while generating splits

toString

public String toString()

Overrides:

toString in class Object

validate

public ByteSource validate()
                    throws IOException

Description copied from interface: ByteSource

Performs validation of the source configuration. This checks things such as the existence and accessibility of the source. It may also optionally rewrite the source to an equivalent one, doing file glob and directory expansion.

Specified by:

validate in interface ByteSource

Returns:

a valid source equivalent to this one

Throws:

IOException - if an I/O error occurs while validating the source