Package | Description |
---|---|
com.actian.dataflow.cal.yarn.server | |
com.actian.dataflow.cal.yarn.shared | |
com.actian.dataflow.operators.io.orc | |
com.actian.dataflow.operators.io.parquet | |
com.pervasive.datarush.aws.s3 | |
com.pervasive.datarush.azure | |
com.pervasive.datarush.cal |
Provides interfaces the define the "cluster abstraction layer".
|
com.pervasive.datarush.gcp | |
com.pervasive.datarush.graphs |
Provides classes and interfaces for the construction of executable
dataflow graphs.
|
com.pervasive.datarush.io |
Provides classes and interfaces performing file-like I/O operations.
|
com.pervasive.datarush.io.compression |
Provides classes and interfaces for supporting data compression.
|
com.pervasive.datarush.io.configuration |
Provides classes and interfaces associated with filesystem configuration
|
com.pervasive.datarush.operators |
Provides classes and interfaces for developing dataflow operators.
|
com.pervasive.datarush.operators.io |
Provides base file I/O components including encoders and decoders.
|
com.pervasive.datarush.operators.io.avro |
Provides operators for reading and writing files in Avro format.
|
com.pervasive.datarush.operators.io.mdf | |
com.pervasive.datarush.operators.io.staging |
Provides operators for reading and writing DataRush staging datasets.
|
com.pervasive.datarush.operators.io.textfile |
Provides operators for reading and writing text data.
|
Modifier and Type | Method and Description |
---|---|
Map<String,Path> |
YarnClusterExecutor.drfsRoots() |
Constructor and Description |
---|
MasterServer(com.pervasive.datarush.hadoop.shims.yarn.AMRMClient amClient,
FileClient client,
Path drfsBaseDir,
JobServerInformation jobWorkerInfo,
JobSpecifier jobSpecifier,
com.pervasive.datarush.cluster.ExecutorOptions execOptions,
List<com.pervasive.datarush.hadoop.shims.yarn.YarnNodeDescriptor> nodeDescriptors,
Path cacheLocation,
Path libArchiveLocation,
String applicationID,
int jobPriority,
com.pervasive.datarush.cluster.preferences.ClusterPreferences clusterPreferences,
Map<String,ApplicationResource> appResources) |
PartitionServer(FileClient client,
Path drfsBaseDir,
String nodeID,
JobServerInformation jobWorkerInfo,
com.pervasive.datarush.cal.dr.NodeAllocationProvider allocationProvider,
JobSpecifier jobSpecifier) |
YarnClusterExecutor(FileClient fileClient,
JobSpecifier jobSpec,
com.pervasive.datarush.hadoop.shims.yarn.ContainerManager containerMgr,
com.pervasive.datarush.cluster.ExecutorOptions execOptions,
Path cacheLocation,
Path libArchiveLocation,
com.pervasive.datarush.cluster.preferences.ClusterPreferences clusterPrefs,
Map<String,ApplicationResource> appResources) |
Modifier and Type | Method and Description |
---|---|
Path |
YarnAppMasterArgs.getCacheLocation() |
Path |
YarnAppMasterArgs.getLibArchiveLocation() |
Modifier and Type | Method and Description |
---|---|
static com.pervasive.datarush.hadoop.shims.yarn.ContainerContext |
ContainerContextBuilder.createContext(com.pervasive.datarush.cluster.Platform targetPlatform,
ClasspathSpecifier classpathSpec,
String jobName,
Class<?> mainClass,
Path archivePath,
com.pervasive.datarush.cluster.preferences.ClusterPreferences clusterPreferences)
Create a container context.
|
Constructor and Description |
---|
DatafileStagingManager(FileClient client,
Path stagingLocation,
GUID jobId) |
YarnAppMasterArgs(JobSpecifier jobSpec,
ModuleConfiguration moduleCfg,
FileMetaConfiguration fileCfg,
ClasspathSpecifier classpathSpec,
com.pervasive.datarush.cluster.ExecutorOptions options,
List<com.pervasive.datarush.hadoop.shims.yarn.YarnNodeDescriptor> nodeDescriptors,
Path cacheLocation,
Path libArchiveLocation,
int jobPriority,
com.pervasive.datarush.cluster.preferences.ClusterPreferences clusterPreferences,
Map<String,ApplicationResource> appResources) |
Constructor and Description |
---|
ReadORC(Path path)
Reads the file specified by the path.
|
Constructor and Description |
---|
ReadParquet(Path path)
Reads the file specified by the path.
|
Modifier and Type | Method and Description |
---|---|
Path |
S3FileSystemProvider.getFileSystemPath(Path path) |
Path |
S3FileSystemProvider.getPath(String path) |
Modifier and Type | Method and Description |
---|---|
FileSystem |
S3FileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
Path |
S3FileSystemProvider.getFileSystemPath(Path path) |
Modifier and Type | Class and Description |
---|---|
class |
AzureFilePath |
Modifier and Type | Method and Description |
---|---|
Path |
AzureFileSystem.createDirectories(Path path) |
Path |
AzureFileSystem.createDirectory(Path path) |
Path |
AzureFileSystemProvider.getFileSystemPath(Path path) |
Path |
AzureFilePath.getParent() |
Path |
AzureFileSystemProvider.getPath(String path) |
Path |
AzureFilePath.resolve(String childPath) |
Path |
AzureFilePath.toAbsolutePath() |
Modifier and Type | Method and Description |
---|---|
Path |
AzureFileSystem.createDirectories(Path path) |
Path |
AzureFileSystem.createDirectory(Path path) |
boolean |
AzureFileSystem.createNewFile(Path path) |
void |
AzureFileSystem.delete(Path path,
boolean recursively) |
boolean |
AzureFileSystem.exists(Path path) |
PathDetails |
AzureFileSystem.getDetails(Path path) |
FileSystem |
AzureFileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
Path |
AzureFileSystemProvider.getFileSystemPath(Path path) |
String |
AzureFileSystem.getFileSystemType(Path path) |
long |
AzureFileSystem.getLength(Path path) |
SplitIterator |
AzureFileSystem.getSplits(Path path,
SplitOptions options) |
boolean |
AzureFileSystem.isCreatable(Path path) |
boolean |
AzureFileSystem.isDirectory(Path path) |
boolean |
AzureFileSystem.isFile(Path path) |
boolean |
AzureFileSystem.isHidden(Path path) |
boolean |
AzureFileSystem.isReadable(Path path) |
boolean |
AzureFileSystem.isWritable(Path path) |
List<PathDetails> |
AzureFileSystem.listDirectory(Path path,
DirectoryFilter filter) |
boolean |
AzureFileSystem.move(FileSystem fromFS,
Path from,
Path to) |
FileChannel |
AzureFileSystem.newFileChannel(Path path) |
InputStream |
AzureFileSystem.newInputStream(Path path) |
OutputStream |
AzureFileSystem.newOutputStream(Path path,
WriteMode mode) |
Modifier and Type | Method and Description |
---|---|
InputStream |
AzureFileSystem.concat(List<Path> paths,
boolean ignoreNonExistent) |
Modifier and Type | Method and Description |
---|---|
Path |
StorageBucket.getMyStorageLocation(NodeAllocationPlan.AssignmentPlan partitionInfo) |
Path |
StorageManager.getMyStorageLocation(StorageBucketId bucketId,
NodeAllocationPlan.AssignmentPlan assignment)
Returns "my" storage location
|
Path |
StorageManagerSpecifier.getWorkerStorageRoot()
Returns the base storage path for this storage manager.
|
Path |
ClasspathSpecifier.Element.path()
Returns the path of this classpath element.
|
Path |
JobSpecifier.storageManagementPath()
Returns the root scratch directory to use.
|
Modifier and Type | Method and Description |
---|---|
List<Path> |
StorageManager.getOtherStorageLocations(StorageBucketId bucketId,
int maxParallelism)
Returns storage locations for all workers up to the specified
maxParallelism . |
List<Path> |
StorageBucket.getWorkerStorageLocations(int maxParallelism) |
Modifier and Type | Method and Description |
---|---|
boolean |
StorageManagerProvider.isProvider(Path storagePath)
Returns whether this provider supports the given storage path
|
ClasspathSpecifier.Element |
ClasspathSpecifier.Element.path(Path path)
Returns a new ClasspathElement, overriding the path.
|
JobSpecifier |
JobSpecifier.storageManagementPath(Path storageManagementPath)
Overrides the storageManagementPath
|
Constructor and Description |
---|
Element(Path path)
Create a new classpath for the given path.
|
Element(Path path,
CacheScopeSpecifier cacheScope,
boolean dynamic,
OSGIOptions osgiOptions)
Create a new classpath for the given path.
|
JobSpecifier(ResourceSpecifier requestedResources,
ClusterSpecifier cluster,
ClasspathSpecifier classpath,
JobInformation jobInformation,
Path storageManagementPath)
Create a new job specifier
|
JobSpecifier(ResourceSpecifier requestedResources,
ClusterSpecifier cluster,
ClasspathSpecifier classpath,
JobInformation jobInformation,
Path storageManagementPath,
String schedulerQueue,
List<String> extensionPaths)
Create a new job specifier
|
Modifier and Type | Class and Description |
---|---|
class |
GcpFilePath |
Modifier and Type | Method and Description |
---|---|
Path |
GcpFileSystem.createDirectories(Path path) |
Path |
GcpFileSystem.createDirectory(Path path) |
Path |
GcpFileSystemProvider.getFileSystemPath(Path path) |
Path |
GcpFilePath.getParent() |
Path |
GcpFileSystemProvider.getPath(String path) |
Path |
GcpFilePath.resolve(String childPath) |
Path |
GcpFilePath.toAbsolutePath() |
Modifier and Type | Method and Description |
---|---|
Path |
GcpFileSystem.createDirectories(Path path) |
Path |
GcpFileSystem.createDirectory(Path path) |
boolean |
GcpFileSystem.createNewFile(Path path) |
void |
GcpFileSystem.delete(Path path,
boolean recursively) |
boolean |
GcpFileSystem.exists(Path path) |
PathDetails |
GcpFileSystem.getDetails(Path path) |
FileSystem |
GcpFileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
Path |
GcpFileSystemProvider.getFileSystemPath(Path path) |
String |
GcpFileSystem.getFileSystemType(Path path) |
long |
GcpFileSystem.getLength(Path path) |
SplitIterator |
GcpFileSystem.getSplits(Path path,
SplitOptions options) |
boolean |
GcpFileSystem.isCreatable(Path path) |
boolean |
GcpFileSystem.isDirectory(Path path) |
boolean |
GcpFileSystem.isFile(Path path) |
boolean |
GcpFileSystem.isHidden(Path path) |
boolean |
GcpFileSystem.isReadable(Path path) |
boolean |
GcpFileSystem.isWritable(Path path) |
List<PathDetails> |
GcpFileSystem.listDirectory(Path path,
DirectoryFilter filter) |
boolean |
GcpFileSystem.move(FileSystem fromFS,
Path from,
Path to) |
FileChannel |
GcpFileSystem.newFileChannel(Path path) |
InputStream |
GcpFileSystem.newInputStream(Path path) |
OutputStream |
GcpFileSystem.newOutputStream(Path path,
WriteMode mode) |
Modifier and Type | Method and Description |
---|---|
InputStream |
GcpFileSystem.concat(List<Path> paths,
boolean ignoreNonExistent) |
Modifier and Type | Field and Description |
---|---|
static EngineProperty<Path> |
EngineConfig.STORAGE_MANAGEMENT_PATH
Property that controls a base path to use for intermediate storage.
|
Modifier and Type | Method and Description |
---|---|
Path |
EngineConfig.getStorageManagementPath()
Returns the root scratch directory to use.
|
Modifier and Type | Method and Description |
---|---|
EngineConfig |
EngineConfig.storageManagementPath(Path path)
Specifies the root scratch directory to use.
|
Modifier and Type | Class and Description |
---|---|
class |
FTPPath |
class |
HadoopFilePath
Implementation of a path for the Hadoop Distributed File System (HDFS).
|
Modifier and Type | Method and Description |
---|---|
static Path |
Paths.asPath(File file)
Converts a
File into a Path object. |
static Path |
Paths.asPath(String path)
Parses a string representing a path into a
Path object. |
static Path[] |
Paths.asPaths(File[] files)
Converts an array of files into an array of paths.
|
Path |
FileSystem.createDirectories(Path path)
Creates the directory identified by the given path,
creating any necessary parent directories.
|
Path |
FileClient.createDirectories(Path path)
Creates the directory identified by the given path,
creating any necessary parent directories.
|
Path |
FileSystem.createDirectory(Path path)
Creates the directory identified by the given path.
|
Path |
FileClient.createDirectory(Path path)
Creates the directory identified by the given path.
|
Path |
FileClient.createTemporaryStorage()
Creates a temporary storage location using default
configuration.
|
Path |
FileClient.createTemporaryStorage(File indexPath,
Path[] storagePaths)
Creates a temporary storage location using the specified
configuration.
|
Path |
HadoopFileSystemProvider.getFileSystemPath(Path path) |
Path |
URLFileSystemProvider.getFileSystemPath(Path path) |
Path |
SFTPFileSystemProvider.getFileSystemPath(Path path) |
Path |
LocalFileSystemProvider.getFileSystemPath(Path path) |
Path |
FileSystemProvider.getFileSystemPath(Path path)
Returns the filesystem root path associated with the given path.
|
Path |
FTPFileSystemProvider.getFileSystemPath(Path path) |
Path |
BuiltinStreamProvider.getFileSystemPath(Path path) |
Path |
HadoopFilePath.getParent() |
Path |
Path.getParent()
Gets the path referring to the location containing
the path.
|
Path |
FTPPath.getParent() |
Path |
PathDetails.getPath()
Gets the path associated with the metadata.
|
Path |
FileSplit.getPath() |
Path |
DataSplit.getPath()
Gets the path to the file on which the
split is defined.
|
Path |
BasicPathDetails.getPath() |
Path |
HadoopFileSystemProvider.getPath(String path) |
Path |
URLFileSystemProvider.getPath(String path) |
Path |
SFTPFileSystemProvider.getPath(String path) |
Path |
LocalFileSystemProvider.getPath(String path) |
Path |
FileSystemProvider.getPath(String path)
Gets the path identified by the given string.
|
Path |
FTPFileSystemProvider.getPath(String path) |
Path |
BuiltinStreamProvider.getPath(String path) |
Path |
FTPPath.getRoot() |
abstract Path |
UnixStyleGlobbing.GlobDefinition.getRootPath()
Gets the initial path from which globbing starts.
|
Path[] |
FileClient.getTempFilesystemRootDirectories()
Returns the root directories used by default for temp storage by this client
|
Path |
HadoopFilePath.resolve(String other) |
Path |
Path.resolve(String childPath)
Adds additional elements to the path.
|
Path |
FTPPath.resolve(String childPath) |
Path |
HadoopFilePath.toAbsolutePath() |
Path |
Path.toAbsolutePath()
Converts a relative path into an absolute
path.
|
Path |
FTPPath.toAbsolutePath() |
Modifier and Type | Method and Description |
---|---|
static List<Path> |
Paths.asPaths(List<String> paths)
Parses a string list representing a path into a list of
Path 's. |
static List<Path> |
Paths.extractPaths(List<PathDetails> details)
Convert a list of
PathDetails objects to a list
of Path objects. |
static List<Path> |
Paths.extractPaths(List<PathDetails> details,
boolean includeHidden)
Convert a list of
PathDetails objects to a list
of Path objects. |
List<Path> |
PathGlob.findMatches() |
List<Path> |
FileClient.listAllFiles(Path path,
DirectoryFilter filter)
Gets the contents of the specified directory including hidden files,
applying a filter.
|
List<Path> |
FileClient.listFiles(Path path)
Gets the contents of the specified directory.
|
List<Path> |
FileClient.listFiles(Path path,
DirectoryFilter filter)
Gets the contents of the specified directory, applying
a filter.
|
List<Path> |
FileClient.matchPaths(String pattern)
Finds all paths matching the specified pattern.
|
Modifier and Type | Method and Description |
---|---|
static HadoopFilePath |
HadoopFilePath.cast(Path p) |
static FileClient |
FileClient.configuredClient(FileMetaConfiguration configuration,
ModuleConfiguration moduleConfiguration,
File indexPathBase,
Path[] tempFsRootDirs)
Creates a new client, authorized according to the given configuration.
|
void |
FileClient.copy(Path from,
Path to)
Copies a file or directory from one location to another.
|
Path |
FileSystem.createDirectories(Path path)
Creates the directory identified by the given path,
creating any necessary parent directories.
|
Path |
FileClient.createDirectories(Path path)
Creates the directory identified by the given path,
creating any necessary parent directories.
|
Path |
FileSystem.createDirectory(Path path)
Creates the directory identified by the given path.
|
Path |
FileClient.createDirectory(Path path)
Creates the directory identified by the given path.
|
boolean |
FileSystem.createNewFile(Path path)
Creates the file identified by the given path, if
it doesn't already exist.
|
boolean |
FileClient.createNewFile(Path path)
Creates the file identified by the given path.
|
Path |
FileClient.createTemporaryStorage(File indexPath,
Path[] storagePaths)
Creates a temporary storage location using the specified
configuration.
|
void |
FileClient.delete(Path path)
Deletes the file or directory identified by the specified
path.
|
void |
FileSystem.delete(Path path,
boolean recursively)
Deletes the file or directory identified by the specified
path.
|
void |
FileClient.delete(Path path,
boolean recursively)
Deletes the file or directory identified by the specified
path.
|
boolean |
FileSystem.exists(Path path)
Indicates whether the specified path represents an
existing file or directory.
|
boolean |
FileClient.exists(Path path)
Indicates whether the specified path represents an
existing file or directory.
|
PathDetails |
FileSystem.getDetails(Path path)
Returns metadata associated with the specified path.
|
PathDetails |
FileClient.getDetails(Path path)
Returns metadata associated with the specified path.
|
FileSystem |
HadoopFileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
FileSystem |
URLFileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
FileSystem |
SFTPFileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
FileSystem |
LocalFileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
FileSystem |
FileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path)
Gets the file system identified by the path.
|
FileSystem |
FTPFileSystemProvider.getFileSystem(FileClient client,
FileConfiguration configuration,
Path path) |
FileSystem |
BuiltinStreamProvider.getFileSystem(FileClient fileClient,
FileConfiguration configuration,
Path path) |
Object |
FileClient.getFileSystemImplementation(Path path)
Returns the underlying implementation object for the file system implementation
of the given path.
|
Path |
HadoopFileSystemProvider.getFileSystemPath(Path path) |
Path |
URLFileSystemProvider.getFileSystemPath(Path path) |
Path |
SFTPFileSystemProvider.getFileSystemPath(Path path) |
Path |
LocalFileSystemProvider.getFileSystemPath(Path path) |
Path |
FileSystemProvider.getFileSystemPath(Path path)
Returns the filesystem root path associated with the given path.
|
Path |
FTPFileSystemProvider.getFileSystemPath(Path path) |
Path |
BuiltinStreamProvider.getFileSystemPath(Path path) |
String |
FileSystem.getFileSystemType(Path path)
Returns an identifier for the filesystem type.
|
String |
FileClient.getFileSystemType(Path path)
Returns an identifier for the filesystem type.
|
static File |
Paths.getIndexFile(Path path)
Returns the index file associated with a temporary file path.
|
long |
FileSystem.getLength(Path path)
Returns the length of the file represented by the given path.
|
long |
FileClient.getLength(Path path)
Returns the length of the file represented by the given path
|
SplitIterator |
FileClient.getSplitIterator(Path path)
Computes data splits over the specified file.
|
SplitIterator |
FileClient.getSplitIterator(Path path,
SplitOptions options)
Computes data splits over the specified file.
|
SplitIterator |
FileSystem.getSplits(Path path,
SplitOptions options)
Computes data splits over the specified file.
|
boolean |
FileSystem.isCreatable(Path path)
Indicates whether the specified path can be created.
|
boolean |
FileClient.isCreatable(Path path)
Indicates whether the specified path can be created.
|
boolean |
FileSystem.isDirectory(Path path)
Indicates whether the specified path represents a directory.
|
boolean |
FileClient.isDirectory(Path path)
Indicates whether the specified path represents a directory.
|
boolean |
FileSystem.isFile(Path path)
Indicates whether the specified path represents a file.
|
boolean |
FileClient.isFile(Path path)
Indicates whether the specified path represents a file.
|
boolean |
FileSystem.isHidden(Path path)
Indicates whether the associated path is hidden by the file system.
|
boolean |
FileClient.isHidden(Path path)
Indicates whether the associated path is hidden by the file system.
|
boolean |
FileSystem.isReadable(Path path)
Indicates whether the specified path can be read.
|
boolean |
FileClient.isReadable(Path path)
Indicates whether the specified path can be read.
|
boolean |
FileSystem.isWritable(Path path)
Indicates whether the specified path can be written.
|
boolean |
FileClient.isWritable(Path path)
Indicates whether the specified path can be written.
|
List<Path> |
FileClient.listAllFiles(Path path,
DirectoryFilter filter)
Gets the contents of the specified directory including hidden files,
applying a filter.
|
List<PathDetails> |
FileClient.listDirectory(Path path)
Gets the contents of the specified directory.
|
List<PathDetails> |
FileSystem.listDirectory(Path path,
DirectoryFilter filter)
Gets the contents of the specified directory, applying
a filter.
|
List<PathDetails> |
FileClient.listDirectory(Path path,
DirectoryFilter filter)
Gets the contents of the specified directory, applying
a filter.
|
List<Path> |
FileClient.listFiles(Path path)
Gets the contents of the specified directory.
|
List<Path> |
FileClient.listFiles(Path path,
DirectoryFilter filter)
Gets the contents of the specified directory, applying
a filter.
|
boolean |
FileSystem.move(FileSystem fromFS,
Path from,
Path to)
Moves a file or directory from one location to
another if possible.
|
void |
FileClient.move(Path from,
Path to)
Moves a file or directory from one location to
another.
|
FileChannel |
FileSystem.newFileChannel(Path path)
Opens the specified file for random access.
|
FileChannel |
FileClient.newFileChannel(Path path)
Opens the specified file for random access.
|
InputStream |
FileSystem.newInputStream(Path path)
Opens the specified file for reading.
|
InputStream |
FileClient.newInputStream(Path path)
Opens the specified file for reading.
|
OutputStream |
FileSystem.newOutputStream(Path path,
WriteMode mode)
Opens the specified file for writing.
|
OutputStream |
FileClient.newOutputStream(Path path,
WriteMode mode)
Opens the specified file for writing.
|
static InputStreamSupplier |
InputStreamSuppliers.path(FileClient client,
Path path)
Returns an InputStreamSupplier that will open the specified path.
|
static InputStreamSupplier |
InputStreamSuppliers.path(FileClient client,
Path path,
boolean ignoreNonExistent)
Returns an InputStreamSupplier that will open the specified path.
|
boolean |
FileClient.supportsDirectories(Path path)
Indicates whether the filesystem corresponding to this path supports directories
|
boolean |
FileClient.supportsRandomAccess(Path path)
Indicates whether the filesystem corresponding to this path supports random access
|
void |
FileClient.unzip(Path from,
Path to)
Recursively unzips the given file to a given directory/file
|
boolean |
FileClient.zipRecursively(Path from,
OutputStream out,
boolean ignoreNonReadable)
Recursively zips the given input file/directory to the given stream
|
Modifier and Type | Method and Description |
---|---|
static InputStreamSupplier |
InputStreamSuppliers.combine(FileClient client,
List<Path> paths,
boolean ignoreNonExistant)
Creates an InputStreamSupplier that will return an input stream consisting of the streams
of each of the sources, combined together.
|
static InputStreamSupplier |
InputStreamSuppliers.concat(FileClient client,
List<Path> paths)
Creates an InputStreamSupplier that will return an input stream consisting of the streams
of each of the paths, concatenated together.
|
static InputStreamSupplier |
InputStreamSuppliers.concat(FileClient client,
List<Path> paths,
boolean ignoreNonExistent)
Creates an InputStreamSupplier that will return an input stream consisting of the streams
of each of the paths, concatenated together.
|
InputStream |
FileSystem.concat(List<Path> paths,
boolean ignoreNonExistent)
Returns an input stream consisting of the given list of paths concatenated together.
|
Constructor and Description |
---|
BasicPathDetails(Path path,
boolean readable,
boolean writable,
boolean hidden,
BasicPathDetails.ObjectType type,
long size) |
CompressedFileSplit(Path path,
CompressionFormat format)
Creates a split encompassing the entire file named by the path.
|
FileSplit(Path path)
Creates a split encompassing the entire file named by the path.
|
FileSplit(Path path,
long start,
long length)
Creates a split of the file named by the path.
|
FileSplit(Path path,
long start,
long length,
FileClient client) |
SplittableCompressedFileSplit(Path path,
long startOffset,
long length,
CompressionFormat format)
Construct an instance.
|
Modifier and Type | Method and Description |
---|---|
static CompressionFormat |
CompressionFormats.detectFormat(Path path)
Locates the identified compression format for a file
based on the path.
|
SplitIterator |
UncompressedData.getSplitIterator(FileClient client,
Path filePath,
SplitOptions options) |
SplitIterator |
SnappyCompression.getSplitIterator(FileClient client,
Path filePath,
SplitOptions options) |
SplitIterator |
GZipCompression.getSplitIterator(FileClient client,
Path filePath,
SplitOptions options) |
SplitIterator |
CompressionFormat.getSplitIterator(FileClient client,
Path filePath,
SplitOptions options)
Create a
SplitIterator for the given file path and options. |
SplitIterator |
BZipCompression.getSplitIterator(FileClient client,
Path filePath,
SplitOptions options) |
Modifier and Type | Method and Description |
---|---|
static FileMetaConfiguration.Entry |
FileMetaConfiguration.entry(Path prefix,
FileConfiguration configuration)
Creates a new entry
|
FileMetaConfiguration.Entry |
FileMetaConfiguration.findEntry(Path path)
Returns the first entry whose prefix is a prefix of the given path
|
FileConfiguration |
FileMetaConfiguration.getConfiguration(Path path)
Returns the configuration to be used for the given path or
FileConfiguration.EMPTY_CONFIGURATION
if none found. |
Modifier and Type | Method and Description |
---|---|
Path |
ExecutionContext.allocateTemporaryStorage()
Allocates space for storing a temporary file or files.
|
Modifier and Type | Method and Description |
---|---|
Path |
BasicByteSource.getPath()
Gets the path identifying the byte source.
|
Path |
BasicByteSink.getPath()
Gets the path identifying the byte sink.
|
Modifier and Type | Method and Description |
---|---|
void |
AbstractReader.setSource(Path path)
Sets the data source to the specified path.
|
void |
AbstractWriter.setTarget(Path path)
Sets the data sink for the writer.
|
Modifier and Type | Method and Description |
---|---|
static ByteSource |
ConcatenatedByteSource.getConcatenatedSource(List<Path> paths)
Creates a source which is the concatenation of the specified paths,
in order.
|
static ByteSource |
ConcatenatedByteSource.getConcatenatedSource(List<Path> paths,
UnreadableSourceAction onUnreadable)
Creates a source which is the concatenation of the specified paths,
in order.
|
Constructor and Description |
---|
AbstractReader(Path path)
Reads the file specified by the path.
|
AbstractWriter(Path path,
WriteMode mode)
Writes to the specified path in the given mode.
|
BasicByteSink(Path path)
Creates a data sink for the named path.
|
BasicByteSink(Path path,
CompressionFormat format)
Creates a data sink for the named path.
|
BasicByteSource(Path path)
Creates a data source for the named path.
|
BasicByteSource(Path path,
CompressionFormat format)
Creates a data source for the named path.
|
BasicByteSource(Path path,
UnreadableSourceAction onUnreadable)
Creates a data source for the named path.
|
Modifier and Type | Method and Description |
---|---|
static AvroMetadata |
AvroMetadata.load(FileClient fileClient,
Path path)
Loads metadata from the specified Avro formatted file.
|
Constructor and Description |
---|
ReadAvro(Path path)
Reads the file specified by the path.
|
WriteAvro(Path path,
WriteMode mode)
Writes to the specified path in the given mode.
|
Constructor and Description |
---|
ReadMDF(Path path)
Reads the file specified by the path.
|
Modifier and Type | Method and Description |
---|---|
Path |
DatasetIterator.getDataset()
loads the next buffer
|
Constructor and Description |
---|
DatasetBuilder(FileClient client,
Path dataset,
int bufferSize,
TokenEncodings baseEncodings,
RecordTokenType type,
WriteMode mode,
int blockSize)
Creates and opens the builder.
|
DatasetIterator(FileClient fileClient,
Path dataset,
int bufferSize,
TokenEncodings baseEncodings)
Creates and opens the iterator.
|
DatasetMetadata(FileClient fileClient,
Path dataset)
Creates a DatasetMetadata for an existing dataset.
|
ReadStagingDataset(Path path)
Reads the file specified by the path as staged data
using default options.
|
WriteStagingDataset(Path path,
WriteMode mode)
Writes to the specified path in the
given mode, using default options.
|
Modifier and Type | Method and Description |
---|---|
JsonTextAnalyzer.JsonAnalysis |
JsonTextAnalyzer.analyze(Path file,
CharsetEncoding charsetSpec)
Analyzes the specified JSON file based on current configuration.
|
FormatAnalyzer.FormatAnalysis |
FormatAnalyzer.analyze(Path file,
CharsetEncoding encoding)
Analyze the specified file expecting the described character set encoding.
|
DelimitedTextAnalyzer.Analysis |
DelimitedTextAnalyzer.analyze(Path file,
CharsetEncoding charsetSpec)
Analyzes the specified file based on current configuration.
|
ARFFAnalyzer.Analysis |
ARFFAnalyzer.analyze(Path file,
CharsetEncoding encoding)
Analyze the specified ARFF file expecting the described character set encoding.
|
JsonTextAnalyzer.JsonAnalysis |
JsonTextAnalyzer.analyze(Path file,
CharsetEncoding charsetSpec,
FileClient client)
Analyzes the specified JSON file based on current configuration.
|
FormatAnalyzer.FormatAnalysis |
FormatAnalyzer.analyze(Path file,
CharsetEncoding encoding,
FileClient client)
Analyze the specified file expecting the described character set encoding.
|
DelimitedTextAnalyzer.Analysis |
DelimitedTextAnalyzer.analyze(Path file,
CharsetEncoding charsetSpec,
FileClient client)
Analyzes the specified file based on current configuration.
|
ARFFAnalyzer.Analysis |
ARFFAnalyzer.analyze(Path file,
CharsetEncoding encoding,
FileClient client)
Analyze the specified ARFF file expecting the described character set encoding.
|
Constructor and Description |
---|
AbstractTextReader(Path path)
Reads the file specified by the path using default options.
|
AbstractTextWriter(Path path,
WriteMode mode)
Writes text to the specified path in the given mode,
using default settings.
|
ReadARFF(Path path)
Reads the file specified by the path as ARFF data
using default options.
|
ReadDelimitedText(Path path)
Reads the file specified by the path as delimited text
using default options.
|
ReadFixedText(Path path)
Reads the file specified by the path as fixed text
using default options.
|
ReadJSON(Path path)
Reads the file specified by the path using default options.
|
ReadLog(Path path)
Reads the file specified by the path as log text
using default options.
|
WriteARFF(Path path,
WriteMode mode)
Writes ARFF data to the specified path in the given mode,
using default settings.
|
WriteDelimitedText(Path path,
WriteMode mode)
Writes delimited text to the specified path in the given mode,
using default settings.
|
WriteFixedText(Path path,
WriteMode mode)
Writes fixed text to the specified path in the given mode,
using default settings.
|
Copyright © 2024 Actian Corporation. All rights reserved.