Module datarush.library
Package com.pervasive.datarush.operators.io.vectorwise

Class LoadVectorOnHadoop

  • All Implemented Interfaces:
    LogicalOperator, RecordSinkOperator, SinkOperator<RecordPort>
    @Deprecated
public class LoadVectorOnHadoop
extends CompositeOperator
implements RecordSinkOperator
    Deprecated.
    this operator has been replaced with LoadActianVector; use that operator instead.
    Bulk load data into the Vector on Hadoop database. This operator is meant to be used distributed within a Hadoop cluster where Vector on Hadoop is running. Data will be written to intermediate files within the Hadoop file system (HDFS). Once all load files are created, the Vector utility vwload is invoked to load the intermediate files from HDFS into Vector on Hadoop. Once the load completes, the intermediate files are removed from HDFS.

    The output of the vwload utility is captured and logged by the operator. If any errors occur, the logged output can be used to determine the issue.

    • Constructor Detail

      • LoadVectorOnHadoop

        public LoadVectorOnHadoop()
        Deprecated.

    • Method Detail

      • getHost

        public String getHost()
        Deprecated.
        Get the server host name property.
        Returns:
        server host name

      • setHost

        public void setHost​(String hostName)
        Deprecated.
        Set the host name property. This is the host name of the server where Vectorwise is installed.
        Parameters:
        hostName - Vectorwise server host name

      • getInstance

        public String getInstance()
        Deprecated.
        Get the Vectorwise instance name.
        Returns:
        instance name

      • setInstance

        public void setInstance​(String instance)
        Deprecated.
        Set the Vectorwise instance name. Defaults to "VW".
        Parameters:
        instanceName - instance name

      • getDatabase

        public String getDatabase()
        Deprecated.
        Get the database name.
        Returns:
        database name

      • setDatabase

        public void setDatabase​(String database)
        Deprecated.
        Set the database name. Specify the database where the target table lives.
        Parameters:
        database - database name

      • getTable

        public String getTable()
        Deprecated.
        Get the target table name.
        Returns:
        target table name

      • setTable

        public void setTable​(String table)
        Deprecated.
        Set the name of the table to load.
        Parameters:
        table - target table name.

      • getUser

        public String getUser()
        Deprecated.
        Get the user account name.
        Returns:
        user account name

      • setUser

        public void setUser​(String userName)
        Deprecated.
        Set the user name.

        When using the vwload load method, the user name is not always needed. The vwload utility has to be executed by the DBA user. The user name and password supplied is for a user account that has write/copy access to the target table.

        Parameters:
        userName - user account name

      • getPassword

        public String getPassword()
        Deprecated.
        Get the password.
        Returns:
        password

      • setPassword

        public void setPassword​(String password)
        Deprecated.
        Set the user's password.

        When using the vwload load method, the password is not always needed. The vwload utility has to be executed by the DBA user. The user name and password supplied is for a user account that has write/copy access to the target table.

        Parameters:
        password -

      • getRenameMapping

        public Map<String,​String> getRenameMapping()
        Deprecated.
        Get the source to target field mapping.
        Returns:
        the rename mapping

      • setRenameMapping

        public void setRenameMapping​(Map<String,​String> renameMapping)
        Deprecated.
        Set a rename mapping. This should be an ordered (i.e. LinkedHashMap) mapping of names. The keys in the map represent the original names in the input record port. The values in the map represent the column names in the table. If the names are the same, the mapping is not required. Any columns that are not included in the mapping will be dropped from the input.

        This is an optional property. If not provided, the input fields are mapped to the target database table by schema order.

        Parameters:
        renameMapping - the mapping from old to new names.

      • getMaxErrors

        public int getMaxErrors()
        Deprecated.
        Get the maximum number of errors allowed.
        Returns:
        maximum errors

      • setMaxErrors

        public void setMaxErrors​(int maxErrors)
        Deprecated.
        Set the maximum number of errors allowed before rolling back the data load operation.
        Parameters:
        maxErrors - maximum errors allowed

      • getRollback

        public boolean getRollback()
        Deprecated.
        Get whether rollback is enabled or disabled.
        Returns:
        rollback setting

      • setRollback

        public void setRollback​(boolean enabled)
        Deprecated.
        Enable or disable rollback processing. If enabled, after the maximum number of errors allowed has been encountered, the data load will be aborted and rolled back. No new data will be inserted into the target table.

        If disabled, the operation is still aborted when the maximum number of errors has been encountered. However, the data load is not rolled back. Any data successfully loaded will appear in the target table.

        Parameters:
        rollback - enable (true) or disable (false) rollback

      • getSshUser

        public String getSshUser()
        Deprecated.
        OS user id used to connect to NameNode of the Hadoop installation. This operator uses this user id to establish SSH connection to machine running the master node of Vector on Hadoop installation
        Returns:
        OS UserId for authenticating the SSH connection

      • setSshUser

        public void setSshUser​(String user)
        Deprecated.
        Set the OS User Id used for authenticating SSH connection to Master Node of Vector on Hadoop
        Parameters:
        user -

      • setSshPassword

        public void setSshPassword​(String passwd)
        Deprecated.
        Set the password for SSH user id
        Parameters:
        password - for user id

      • getSshPassword

        public String getSshPassword()
        Deprecated.
        Password for SSH User id
        Returns:
        password for SSH user

      • getInitializeTableSQL

        public String getInitializeTableSQL()
        Deprecated.
        Retrieves the SQL statement to execute before processing any records.
        Returns:
        the SQL statement to execute before processing any records

      • setInitializeTableSQL

        public void setInitializeTableSQL​(String initializeTableSQL)
        Deprecated.
        Sets the SQL statement to execute before processing any records. If the table does not exist this value is required, and should contain a CREATE TABLE statement to create the table.

        This statement is executed only once, regardless of #partitionCount.

        Parameters:
        initializeTableSQL - the SQL statement to execute before processing any records

      • getFinalizeTableSQL

        public String getFinalizeTableSQL()
        Deprecated.
        Retrieves the SQL statement to execute after processing all the records.
        Returns:
        the SQL statement to execute after processing all the records

      • setFinalizeTableSQL

        public void setFinalizeTableSQL​(String finalizeTableSQL)
        Deprecated.
        Sets the SQL statement to execute after processing all the records. One example use is a CREATE INDEX statement.

        This statement is executed only once, regardless of #partitionCount.

        Parameters:
        finalizeTableSQL - the SQL statement to execute after processing all the records

      • getTmpDirectory

        public String getTmpDirectory()
        Deprecated.

      • setTmpDirectory

        public void setTmpDirectory​(String tmpDirectory)
        Deprecated.
        Set the temporary directory that is to be used for storing the intermediate loader files. The directory must exist in the HDFS filesystem on the target Hadoop cluster where Vector is running to be used by the distributed loader.
        Parameters:
        tmpDirectory - path to the temporary directory

      • setCharset

        public void setCharset​(String charsetName)
        Deprecated.
        Set the character set used for data staging and loading for vwload load method.
        Parameters:
        charsetName - character set used when staging and loading for vwload method

      • getCharset

        public String getCharset()
        Deprecated.
        Get the character set used when staging and loading for vwload method for Vectorwise operator.
        Returns:
        charset used when staging and loading data for vwload method

      • compose

        protected void compose​(CompositionContext ctx)
        Deprecated.
        Description copied from class: CompositeOperator
        Compose the body of this operator. Implementations should do the following:
        1. Perform any validation of configuration, input types, etc
        2. Instantiate and configure sub-operators, adding them to the provided context via the method OperatorComposable.add(O)
        3. Create necessary connections via the method OperatorComposable.connect(P, P). This includes connections from the composite's input ports to sub-operators, connections between sub-operators, and connections from sub-operators output ports to the composite's output ports
        Specified by:
        compose in class CompositeOperator
        Parameters:
        ctx - the context