Module datarush.commons
Package com.pervasive.datarush.io

Interface FileSystem

  • All Known Implementing Classes:
    AzureFileSystem, GcpFileSystem
    public interface FileSystem
    Describes the file system identified by a path scheme. A Path points to a location (existing or not) on a FileSystem.

    A FileSystem is rarely used directly. One only needs to be creating a new path scheme, in which case a FileSystemProvider must also be implemented.

    • Method Detail

      • getFileSystemType

        String getFileSystemType​(Path path)
        Returns an identifier for the filesystem type. This identifier is used for reporting statistics to be able to distinguish local from non-local io.
        Parameters:
        path - the path
        Returns:
        a unique identifier

      • getProvider

        FileSystemProvider getProvider()
        Gets the provider for the path scheme associated with the file system.
        Returns:
        the associated provider object

      • supportsRandomAccess

        boolean supportsRandomAccess()
        Indicates whether the file system supports random access to files.

        If a file system does not support random access, calling FileClient.newFileChannel(Path) will raise an error for paths referring to the file system.

        Returns:
        true if random access is supported, false otherwise.

      • supportsDirectories

        boolean supportsDirectories()
        Indicates whether the file system supports directories.
        Returns:
        true if directories are supported, false otherwise.

      • matchPaths

        List<PathDetails> matchPaths​(String pattern)
                      throws IOException
        Gets all paths which match the given pattern. Patterns may be scheme-specific; refer to the provider implementation of a specific scheme to determine valid pattern syntax.
        Parameters:
        pattern - the file pattern to use
        Returns:
        all paths matching the pattern
        Throws:
        IOException - if I/O errors occur
        IllegalArgumentException - if the pattern is not prefixed with a scheme supported by the provider

      • createDirectories

        Path createDirectories​(Path path)
                throws IOException
        Creates the directory identified by the given path, creating any necessary parent directories.
        Parameters:
        path - the directory to create
        Returns:
        the path to the directory
        Throws:
        IOException - if I/O errors occur
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • createDirectory

        Path createDirectory​(Path path)
              throws IOException
        Creates the directory identified by the given path. All parent directories must already exist. Use createDirectories(Path) if nonexistent parents need to be created.
        Parameters:
        path - the directory to create
        Returns:
        the path to the directory
        Throws:
        IOException - if I/O errors occur or if the parent directory does not exist
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • createNewFile

        boolean createNewFile​(Path path)
               throws IOException
        Creates the file identified by the given path, if it doesn't already exist. All parent directories must already exist. Use createDirectories(Path) if nonexistent parents need to be created.
        Parameters:
        path - the directory to create
        Returns:
        true if the file was created, false if it already existed.
        Throws:
        IOException - if I/O errors occur or if the parent directory does not exist
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • delete

        void delete​(Path path,
            boolean recursively)
     throws IOException
        Deletes the file or directory identified by the specified path. If the path is a directory, the contents will be deleted recursively as indicated. It is not an error to delete a non-existent path.
        Parameters:
        path - the file or directory to delete
        recursively - indicates whether deletes of directories should recursively delete the contents
        Throws:
        IOException - if I/O errors occur or if the path identifies a non-empty directory and a recursive delete was not requested
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • move

        boolean move​(FileSystem fromFS,
             Path from,
             Path to)
      throws IOException
        Moves a file or directory from one location to another if possible.
        Parameters:
        fromFS - the source filesystem
        from - the source location
        to - the target location
        Returns:
        true if a move was possible (otherwise caller should fall back to copy-and-delete)
        Throws:
        IOException - if I/O errors occur

      • newInputStream

        InputStream newInputStream​(Path path)
                    throws IOException
        Opens the specified file for reading. If the path identifies a directory, all files in the directory are read as a concatenated stream, in the order they are returned by #listFiles(Path). The read starts at the first byte of the (first) file. The returned stream is generally not buffered; consult specific implementations for details.
        Parameters:
        path - the file or directory to read
        Returns:
        a stream for reading the contents of the path
        Throws:
        IOException - if I/O errors occur
        FileNotFoundException - if the specified path does not exist
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • newOutputStream

        OutputStream newOutputStream​(Path path,
                             WriteMode mode)
                      throws IOException
        Opens the specified file for writing. Behavior depends on the provided mode and whether the target file exists. If the target file does not exist, it is created and the resulting stream is positioned at the first byte. Otherwise, if the target file exists:
        • If the mode is CREATE_NEW, an error is raised.
        • If the mode is OVERWRITE, the existing file is replaced with the byte written to the resulting stream.
        • If the mode is APPEND, the resulting stream is positioned after the last byte of the existing file.

        The returned stream is generally not buffered; consult specific implementations for details.

        Parameters:
        path - the file to write
        mode - how to handle writes to existing files
        Returns:
        a stream for writing (or appending) the contents of the file
        Throws:
        IOException - if I/O errors occur or if the target is a directory
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • newFileChannel

        FileChannel newFileChannel​(Path path)
                    throws IOException
        Opens the specified file for random access. Not all schemes support random access; consult specific implementations for details.
        Parameters:
        path - the file or directory to read
        Returns:
        a stream for reading the contents of the path
        Throws:
        IOException - if I/O errors occur
        FileNotFoundException - if the specified path does not exist
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • exists

        boolean exists​(Path path)
        throws IOException
        Indicates whether the specified path represents an existing file or directory.
        Parameters:
        path - the path to test
        Returns:
        true the path exists, false otherwise.
        Throws:
        IOException - if I/O errors occur
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • getDetails

        PathDetails getDetails​(Path path)
                throws IOException
        Returns metadata associated with the specified path.
        Parameters:
        path - the path for which to get metadata
        Returns:
        metadata for the specified file system object, if it exists. null if it does not exist.
        Throws:
        IOException - if I/O errors occur

      • getLength

        long getLength​(Path path)
        throws IOException
        Returns the length of the file represented by the given path.
        Parameters:
        path - the path to test
        Returns:
        the length of the file
        Throws:
        IOException - if I/O errors occur

      • isDirectory

        boolean isDirectory​(Path path)
             throws IOException
        Indicates whether the specified path represents a directory.
        Parameters:
        path - the path to test
        Returns:
        true the path represents a directory,
        Throws:
        IOException

      • isFile

        boolean isFile​(Path path)
        throws IOException
        Indicates whether the specified path represents a file.
        Parameters:
        path - the path to test
        Returns:
        true the path represents a file,
        Throws:
        IOException

      • isReadable

        boolean isReadable​(Path path)
            throws IOException
        Indicates whether the specified path can be read.
        Parameters:
        path - the path to test
        Returns:
        true the path represents a readable file or directory, false otherwise.
        Throws:
        IOException - if I/O errors occur
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • isWritable

        boolean isWritable​(Path path)
            throws IOException
        Indicates whether the specified path can be written.
        Parameters:
        path - the path to test
        Returns:
        true the path represents a readable file or directory, false otherwise.
        Throws:
        IOException - if I/O errors occur
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • isCreatable

        boolean isCreatable​(Path path)
             throws IOException
        Indicates whether the specified path can be created.
        Parameters:
        path - the path to test
        Returns:
        true the path represents a nonexistent file or directory which could be created, false otherwise.
        Throws:
        IOException - if I/O errors occur
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • isHidden

        boolean isHidden​(Path path)
          throws IOException
        Indicates whether the associated path is hidden by the file system.
        Parameters:
        path - the path to test
        Returns:
        true if the path is hidden, false otherwise.
        Throws:
        IOException - if I/O errors occur
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • listDirectory

        List<PathDetails> listDirectory​(Path path,
                                DirectoryFilter filter)
                         throws IOException
        Gets the contents of the specified directory, applying a filter.
        Parameters:
        path - the directory for which to get a filtered content list
        filter - a filter to apply to the contents
        Returns:
        all children of the directory which pass the filter
        Throws:
        IOException - if I/O errors occur or if the path identifies a file
        IllegalArgumentException - if the path is not is not for a scheme supported by the provider

      • getSplits

        SplitIterator getSplits​(Path path,
                        SplitOptions options)
                 throws IOException
        Computes data splits over the specified file. The supplied options are used to control the division of the file into splits, if the file system supports it.

        If a provider can provide locality information, it should, as this can be used to guide distributed execution.

        Parameters:
        path - the file for which to get splits
        options - configuration for the process of dividing the file
        Returns:
        an iterator generating the data splits over the target
        Throws:
        IOException - if an I/O error occurs

      • supportsConcat

        boolean supportsConcat()
        Returns whether this filesystem supports the concat method.
        Returns:
        whether this filesystem supports the concat method

      • concat

        InputStream concat​(List<Path> paths,
                   boolean ignoreNonExistent)
            throws IOException
        Returns an input stream consisting of the given list of paths concatenated together. Most filesystem implementations should not need to implement this method. This is an optimization for remote filesystems to perform a bulk-fetch.
        Parameters:
        paths - the list of paths
        ignoreNonExistent - whether to skip non-existent files
        Returns:
        a concatenated input stream
        Throws:
        IOException
        UnsupportedOperationException - If this filesystem does not support concatenation.

      • getImplementation

        Object getImplementation()