com.pervasive.datarush.operators.io.vectorwise.dl

Class LoadVectorOnHadoopDirect

java.lang.Object

com.pervasive.datarush.operators.AbstractLogicalOperator

com.pervasive.datarush.operators.CompositeOperator

com.pervasive.datarush.operators.io.vectorwise.dl.LoadVectorOnHadoopDirect

All Implemented Interfaces:

LogicalOperator

public final class LoadVectorOnHadoopDirect
extends CompositeOperator

Load data from the data input stream into an Actian Vector on Hadoop database table. Input fields are mapped to output columns by name. Any unmapped input fields are ignored. Unmapped table columns are not loaded and must be nullable. The Actian Vector on Hadoop database supports the concept of streams. A stream represents a unit of parallelism within the distributed database. The loader operator communicates directly with streams to transfer data into the target database. The input data is not repartitioned to match the number of target streams. If the parallelism (partition count) of the loader operator is less than the number of database streams, some streams will be left idle during the load process.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DEFAULT_VECTOR_H_INSTANCE
static int	DEFAULT_VECTOR_SIZE

Constructor Summary

Constructors

Constructor and Description
LoadVectorOnHadoopDirect()

Method Summary

Modifier and Type	Method and Description
protected void	compose(CompositionContext context) Compose the body of this operator.
String	getDatabase() Get the database name.
String	getHost() Get the host name of the server where the leader node is running.
RecordPort	getInputPort()
String	getInsertMode() Get insert mode ( DEFAULT/ROW/BULK).
String	getInstance() Get the Vector instance name.
int	getMaxErrors() Get the maximum number of errors allowed.
String	getPassword() Get user's password.
Map<String,String>	getRenameMapping() Get the source to target field mapping.
boolean	getRollback() Get whether rollback is enabled or disabled.
String	getTable() Get the target table name.
String	getUser() Get the user account name.
int	getVectorSize() Get the buffer size (in rows).
void	setDatabase(String database) Set the database name.
void	setHost(String host) Set the host name of the server where the leader node is running.
void	setInsertMode(String insertMode) Set insert mode statement applies for Insert and Merge operation.
void	setInstance(String instance) Set the Vector instance name.
void	setMaxErrors(int maxErrors) Set the maximum number of errors allowed per stream before rolling back the data load operation.
void	setPassword(String password) Set user's password.
void	setRenameMapping(Map<String,String> renameMapping) Set a rename mapping.
void	setRollback(boolean enabled) Enable or disable rollback processing.
void	setTable(String table) Set the name of the table to load.
void	setUser(String user) Set the user account name.
void	setVectorSize(int vectorSize) Set the size of the buffer (in rows) used to cache data before sending to the Vector engine.

Methods inherited from class com.pervasive.datarush.operators.AbstractLogicalOperator

disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_VECTOR_SIZE

public static final int DEFAULT_VECTOR_SIZE

See Also:

Constant Field Values

DEFAULT_VECTOR_H_INSTANCE

public static final String DEFAULT_VECTOR_H_INSTANCE

See Also:

Constant Field Values

Constructor Detail

LoadVectorOnHadoopDirect

public LoadVectorOnHadoopDirect()

Method Detail

getUser

public String getUser()

Get the user account name.

Returns:

user account name

setUser

public void setUser(String user)

Set the user account name.

Parameters:

user - the name of the user account that has write/copy permissions

getPassword

public String getPassword()

Get user's password.

Returns:

user's password

setPassword

public void setPassword(String password)

Set user's password.

Parameters:

password - the password of the user account having write/copy permissions

getRenameMapping

public Map<String,String> getRenameMapping()

Get the source to target field mapping.

Returns:

the rename mapping

setRenameMapping

public void setRenameMapping(Map<String,String> renameMapping)

Set a rename mapping. This should be an ordered (i.e. LinkedHashMap) mapping of names. The keys in the map represent the original names in the input record port. The values in the map represent the column names in the table. If the names are the same, the mapping is not required. Any columns that are not included in the mapping will be dropped from the input.

This is an optional property. If not provided, the input fields are mapped to the target database table by schema order.

Parameters:

renameMapping - the mapping from old to new names.

getMaxErrors

public int getMaxErrors()

Get the maximum number of errors allowed.

Returns:

maximum errors

setMaxErrors

public void setMaxErrors(int maxErrors)

Set the maximum number of errors allowed per stream before rolling back the data load operation.

Parameters:

maxErrors - maximum errors allowed per stream

getRollback

public boolean getRollback()

Get whether rollback is enabled or disabled.

Returns:

rollback setting

setRollback

public void setRollback(boolean enabled)

Enable or disable rollback processing. If enabled, after the maximum number of errors allowed has been encountered, the data load will be aborted and rolled back. No new data will be inserted into the target table.

If disabled, the operation is still aborted when the maximum number of errors has been encountered. However, the data load is not rolled back. Any data successfully loaded will appear in the target table.

Parameters:

rollback - enable (true) or disable (false) rollback

getHost

public String getHost()

Get the host name of the server where the leader node is running.

Returns:

server host name of the leader node

setHost

public void setHost(String host)

Set the host name of the server where the leader node is running.

Parameters:

host - the server host name of the leader node

getInstance

public String getInstance()

Get the Vector instance name.

Returns:

instance name

setInstance

public void setInstance(String instance)

Set the Vector instance name. Defaults to "VH".

Parameters:

instance - the instance name

getDatabase

public String getDatabase()

Get the database name.

Returns:

database name

setDatabase

public void setDatabase(String database)

Set the database name. Specify the database where the target table lives.

Parameters:

database - the database name

getTable

public String getTable()

Get the target table name.

Returns:

target table name

setTable

public void setTable(String table)

Set the name of the table to load.

Parameters:

table - the name of the target table.

getVectorSize

public int getVectorSize()

Get the buffer size (in rows).

Returns:

buffer size

setVectorSize

public void setVectorSize(int vectorSize)

Set the size of the buffer (in rows) used to cache data before sending to the Vector engine. Defaults to 1024.

Parameters:

vectorSize - buffer size in rows

getInputPort

public RecordPort getInputPort()

getInsertMode

public String getInsertMode()

Get insert mode ( DEFAULT/ROW/BULK).

Returns:

insertMode

setInsertMode

public void setInsertMode(String insertMode)

Set insert mode statement applies for Insert and Merge operation. Default insert mode is set to DEFAULT.

Parameters:

insertMode - insert mode for inserts and merges.

compose

protected void compose(CompositionContext context)

Description copied from class: CompositeOperator

Compose the body of this operator. Implementations should do the following:

1. Perform any validation of configuration, input types, etc
2. Instantiate and configure sub-operators, adding them to the provided context via the method OperatorComposable.add(O)
3. Create necessary connections via the method OperatorComposable.connect(P, P). This includes connections from the composite's input ports to sub-operators, connections between sub-operators, and connections from sub-operators output ports to the composite's output ports

Specified by:

compose in class CompositeOperator

Parameters:

context - the context