Module datarush.azure

Package com.pervasive.datarush.azure

Class AzureFileSplit

java.lang.Object

com.pervasive.datarush.io.FileSplit

com.pervasive.datarush.azure.AzureFileSplit

All Implemented Interfaces:

DataSplit, Serializable

public class AzureFileSplit
extends FileSplit

Describes a range of bytes from an Azure Blob. Ranges are identified by a start offset and a length. DataSplit objects are used to describe how files can be divided into pieces which can then be parsed in parallel.

See Also:

• Serialized Form

Constructor Summary

Constructors

Constructor	Description
AzureFileSplit(AzureFilePath path)
AzureFileSplit(AzureFilePath path, long start, long length)
AzureFileSplit(AzureFilePath path, long start, long length, FileClient client)

Method Summary

Modifier and Type	Method	Description
AzureFileSplit	authorize(FileClient client)	Creates an identical split which will use the specified authorization context for access.
InputStream	openSource()	Opens the underlying source for access.
SplitInputStream	openSplit(int buffer)	Opens the split for reading using the specified size for the read buffer.

Methods inherited from class com.pervasive.datarush.io.FileSplit

getEndOffset, getFileClient, getLength, getPath, getStartOffset, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

AzureFileSplit

public AzureFileSplit(AzureFilePath path)

AzureFileSplit

public AzureFileSplit(AzureFilePath path,
 long start,
 long length)

AzureFileSplit

public AzureFileSplit(AzureFilePath path,
 long start,
 long length,
 FileClient client)

Method Details

openSplit

public SplitInputStream openSplit(int buffer)
                           throws IOException

Description copied from interface: DataSplit

Opens the split for reading using the specified size for the read buffer. The reader will initially be positioned at the first byte of the split. The reader will indicate when the last byte of the split has been read via SplitInputStreamImpl.hasOverrun().

Specified by:

openSplit in interface DataSplit

Overrides:

openSplit in class FileSplit

Parameters:

buffer - the size of the buffer to use for reads, in bytes

Returns:

a reader of the data in the split

Throws:

IOException - if an I/O error occurs opening the underlying source

openSource

public InputStream openSource()
                       throws IOException

Description copied from interface: DataSplit

Opens the underlying source for access. Initially, the stream is positioned at the first byte of the source. Unlike DataSplit.openSplit(int), the caller is responsible for making sure accesses are aligned to split boundaries. The stream is also unbuffered.

This method may be required for dealing with formats which store metadata at the beginning of the file.

Specified by:

openSource in interface DataSplit

Overrides:

openSource in class FileSplit

Returns:

a reader of the data in the underlying source

Throws:

IOException - if an I/O error occurs opening the underlying source

authorize

public AzureFileSplit authorize(FileClient client)

Description copied from interface: DataSplit

Creates an identical split which will use the specified authorization context for access.

This method is used by clients of the IO APIs which want to provide an alternative to the OS-level authorization inherited from the JVM's execution environment. Data access methods for the split will use the supplied context.

The authorization context is not a serializable attribute of a data split, as it represents the environment in which the data in accesses, not a property of the data itself. The context is associated with the split as a matter of convenience.

Specified by:

authorize in interface DataSplit

Overrides:

authorize in class FileSplit

Parameters:

client - the authorization context to use for access

Returns:

a split using the provided authorization context