com.pervasive.datarush.operators.io.vectorwise

Class LoadVectorOnHadoop

java.lang.Object

com.pervasive.datarush.operators.AbstractLogicalOperator

com.pervasive.datarush.operators.CompositeOperator

com.pervasive.datarush.operators.io.vectorwise.LoadVectorOnHadoop

All Implemented Interfaces:

LogicalOperator, RecordSinkOperator, SinkOperator<RecordPort>

Deprecated.

this operator has been replaced with LoadActianVector; use that operator instead.

@Deprecated
public class LoadVectorOnHadoop
extends CompositeOperator
implements RecordSinkOperator

Bulk load data into the Vector on Hadoop database. This operator is meant to be used distributed within a Hadoop cluster where Vector on Hadoop is running. Data will be written to intermediate files within the Hadoop file system (HDFS). Once all load files are created, the Vector utility vwload is invoked to load the intermediate files from HDFS into Vector on Hadoop. Once the load completes, the intermediate files are removed from HDFS.

The output of the vwload utility is captured and logged by the operator. If any errors occur, the logged output can be used to determine the issue.

Constructor Summary

Constructors

Constructor and Description
LoadVectorOnHadoop() Deprecated.

Method Summary

Modifier and Type	Method and Description
protected void	compose(CompositionContext ctx) Deprecated. Compose the body of this operator.
String	getCharset() Deprecated. Get the character set used when staging and loading for vwload method for Vectorwise operator.
String	getDatabase() Deprecated. Get the database name.
String	getFinalizeTableSQL() Deprecated. Retrieves the SQL statement to execute after processing all the records.
String	getHost() Deprecated. Get the server host name property.
String	getInitializeTableSQL() Deprecated. Retrieves the SQL statement to execute before processing any records.
RecordPort	getInput() Deprecated. Gets the record port providing the input data to the sink.
String	getInstance() Deprecated. Get the Vectorwise instance name.
int	getMaxErrors() Deprecated. Get the maximum number of errors allowed.
String	getPassword() Deprecated. Get the password.
Map<String,String>	getRenameMapping() Deprecated. Get the source to target field mapping.
boolean	getRollback() Deprecated. Get whether rollback is enabled or disabled.
String	getSshPassword() Deprecated. Password for SSH User id
String	getSshUser() Deprecated. OS user id used to connect to NameNode of the Hadoop installation.
String	getTable() Deprecated. Get the target table name.
String	getTmpDirectory() Deprecated.
String	getUser() Deprecated. Get the user account name.
void	setCharset(String charsetName) Deprecated. Set the character set used for data staging and loading for vwload load method.
void	setDatabase(String database) Deprecated. Set the database name.
void	setFinalizeTableSQL(String finalizeTableSQL) Deprecated. Sets the SQL statement to execute after processing all the records.
void	setHost(String hostName) Deprecated. Set the host name property.
void	setInitializeTableSQL(String initializeTableSQL) Deprecated. Sets the SQL statement to execute before processing any records.
void	setInstance(String instance) Deprecated. Set the Vectorwise instance name.
void	setMaxErrors(int maxErrors) Deprecated. Set the maximum number of errors allowed before rolling back the data load operation.
void	setPassword(String password) Deprecated. Set the user's password.
void	setRenameMapping(Map<String,String> renameMapping) Deprecated. Set a rename mapping.
void	setRollback(boolean enabled) Deprecated. Enable or disable rollback processing.
void	setSshPassword(String passwd) Deprecated. Set the password for SSH user id
void	setSshUser(String user) Deprecated. Set the OS User Id used for authenticating SSH connection to Master Node of Vector on Hadoop
void	setTable(String table) Deprecated. Set the name of the table to load.
void	setTmpDirectory(String tmpDirectory) Deprecated. Set the temporary directory that is to be used for storing the intermediate loader files.
void	setUser(String userName) Deprecated. Set the user name.

Methods inherited from class com.pervasive.datarush.operators.AbstractLogicalOperator

disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.pervasive.datarush.operators.LogicalOperator

disableParallelism, getInputPorts, getOutputPorts

Constructor Detail

LoadVectorOnHadoop

public LoadVectorOnHadoop()

Deprecated.

Method Detail

getInput

public RecordPort getInput()

Deprecated.

Description copied from interface: RecordSinkOperator

Gets the record port providing the input data to the sink.

Specified by:

getInput in interface RecordSinkOperator

Specified by:

getInput in interface SinkOperator<RecordPort>

Returns:

the input port for the sink

getHost

public String getHost()

Deprecated.

Get the server host name property.

Returns:

server host name

setHost

public void setHost(String hostName)

Deprecated.

Set the host name property. This is the host name of the server where Vectorwise is installed.

Parameters:

hostName - Vectorwise server host name

getInstance

public String getInstance()

Deprecated.

Get the Vectorwise instance name.

Returns:

instance name

setInstance

public void setInstance(String instance)

Deprecated.

Set the Vectorwise instance name. Defaults to "VW".

Parameters:

instanceName - instance name

getDatabase

public String getDatabase()

Deprecated.

Get the database name.

Returns:

database name

setDatabase

public void setDatabase(String database)

Deprecated.

Set the database name. Specify the database where the target table lives.

Parameters:

database - database name

getTable

public String getTable()

Deprecated.

Get the target table name.

Returns:

target table name

setTable

public void setTable(String table)

Deprecated.

Set the name of the table to load.

Parameters:

table - target table name.

getUser

public String getUser()

Deprecated.

Get the user account name.

Returns:

user account name

setUser

public void setUser(String userName)

Deprecated.

Set the user name.

When using the vwload load method, the user name is not always needed. The vwload utility has to be executed by the DBA user. The user name and password supplied is for a user account that has write/copy access to the target table.

Parameters:

userName - user account name

getPassword

public String getPassword()

Deprecated.

Get the password.

Returns:

password

setPassword

public void setPassword(String password)

Deprecated.

Set the user's password.

When using the vwload load method, the password is not always needed. The vwload utility has to be executed by the DBA user. The user name and password supplied is for a user account that has write/copy access to the target table.

Parameters:

password -

getRenameMapping

public Map<String,String> getRenameMapping()

Deprecated.

Get the source to target field mapping.

Returns:

the rename mapping

setRenameMapping

public void setRenameMapping(Map<String,String> renameMapping)

Deprecated.

Set a rename mapping. This should be an ordered (i.e. LinkedHashMap) mapping of names. The keys in the map represent the original names in the input record port. The values in the map represent the column names in the table. If the names are the same, the mapping is not required. Any columns that are not included in the mapping will be dropped from the input.

This is an optional property. If not provided, the input fields are mapped to the target database table by schema order.

Parameters:

renameMapping - the mapping from old to new names.

getMaxErrors

public int getMaxErrors()

Deprecated.

Get the maximum number of errors allowed.

Returns:

maximum errors

setMaxErrors

public void setMaxErrors(int maxErrors)

Deprecated.

Set the maximum number of errors allowed before rolling back the data load operation.

Parameters:

maxErrors - maximum errors allowed

getRollback

public boolean getRollback()

Deprecated.

Get whether rollback is enabled or disabled.

Returns:

rollback setting

setRollback

public void setRollback(boolean enabled)

Deprecated.

Enable or disable rollback processing. If enabled, after the maximum number of errors allowed has been encountered, the data load will be aborted and rolled back. No new data will be inserted into the target table.

If disabled, the operation is still aborted when the maximum number of errors has been encountered. However, the data load is not rolled back. Any data successfully loaded will appear in the target table.

Parameters:

rollback - enable (true) or disable (false) rollback

getSshUser

public String getSshUser()

Deprecated.

OS user id used to connect to NameNode of the Hadoop installation. This operator uses this user id to establish SSH connection to machine running the master node of Vector on Hadoop installation

Returns:

OS UserId for authenticating the SSH connection

setSshUser

public void setSshUser(String user)

Deprecated.

Set the OS User Id used for authenticating SSH connection to Master Node of Vector on Hadoop

Parameters:

user -

setSshPassword

public void setSshPassword(String passwd)

Deprecated.

Set the password for SSH user id

Parameters:

password - for user id

getSshPassword

public String getSshPassword()

Deprecated.

Password for SSH User id

Returns:

password for SSH user

getInitializeTableSQL

public String getInitializeTableSQL()

Deprecated.

Retrieves the SQL statement to execute before processing any records.

Returns:

the SQL statement to execute before processing any records

setInitializeTableSQL

public void setInitializeTableSQL(String initializeTableSQL)

Deprecated.

Sets the SQL statement to execute before processing any records. If the table does not exist this value is required, and should contain a CREATE TABLE statement to create the table.

This statement is executed only once, regardless of #partitionCount.

Parameters:

initializeTableSQL - the SQL statement to execute before processing any records

getFinalizeTableSQL

public String getFinalizeTableSQL()

Deprecated.

Retrieves the SQL statement to execute after processing all the records.

Returns:

the SQL statement to execute after processing all the records

setFinalizeTableSQL

public void setFinalizeTableSQL(String finalizeTableSQL)

Deprecated.

Sets the SQL statement to execute after processing all the records. One example use is a CREATE INDEX statement.

This statement is executed only once, regardless of #partitionCount.

Parameters:

finalizeTableSQL - the SQL statement to execute after processing all the records

getTmpDirectory

public String getTmpDirectory()

Deprecated.

setTmpDirectory

public void setTmpDirectory(String tmpDirectory)

Deprecated.

Set the temporary directory that is to be used for storing the intermediate loader files. The directory must exist in the HDFS filesystem on the target Hadoop cluster where Vector is running to be used by the distributed loader.

Parameters:

tmpDirectory - path to the temporary directory

setCharset

public void setCharset(String charsetName)

Deprecated.

Set the character set used for data staging and loading for vwload load method.

Parameters:

charsetName - character set used when staging and loading for vwload method

getCharset

public String getCharset()

Deprecated.

Get the character set used when staging and loading for vwload method for Vectorwise operator.

Returns:

charset used when staging and loading data for vwload method

compose

protected void compose(CompositionContext ctx)

Deprecated.

Description copied from class: CompositeOperator

Compose the body of this operator. Implementations should do the following:

1. Perform any validation of configuration, input types, etc
2. Instantiate and configure sub-operators, adding them to the provided context via the method OperatorComposable.add(O)
3. Create necessary connections via the method OperatorComposable.connect(P, P). This includes connections from the composite's input ports to sub-operators, connections between sub-operators, and connections from sub-operators output ports to the composite's output ports

Specified by:

compose in class CompositeOperator

Parameters:

ctx - the context