Module datarush.commons
Package com.pervasive.datarush.cal

Interface ClusterProvider

All Known Implementing Classes:
YarnClusterProvider
public interface ClusterProvider
Extension point to register cluster providers for new schemes. ClusterProvider must be registered by creating a file META-INF/services/com.pervasive.datarush.cal.ClusterProvider that contains the implementation class.

  • Method Details

    • isProviderFor

      boolean isProviderFor(ClusterSpecifier spec)
      Returns whether this provider is the correct provider for the given specifier.
      Parameters:
      spec - the specifier
      Returns:
      whether this provider knows how to handle the given specifier

    • validateConnection

      void validateConnection(FileClient fileClient, ClusterSpecifier spec)
      Gives the provider an opportunity to validate the specifier
      Parameters:
      spec - the specifier

    • getDefaultStorageManagementPath

      String getDefaultStorageManagementPath(ClusterSpecifier spec)
      Returns the default root path for this cluster provider. For example, within the DR cluster, the default path is drfs://clustermanager:port.
      Parameters:
      spec - the cluster specifier
      Returns:
      the default root path

    • createExecutor

      JobExecutorService createExecutor(FileClient fileClient, JobSpecifier spec)
      Create a new job executor service.
      Parameters:
      fileClient - the file client to use
      spec - the job specifier
      Returns:
      a new job executor service

    • getJobHistoryService

      JobHistoryService getJobHistoryService(FileClient client, ClusterSpecifier spec)