com.actian.vector.transfer

Class VectorDataTransfer

java.lang.Object

com.actian.vector.transfer.VectorDataTransfer

public class VectorDataTransfer
extends Object

The JNI interface that wraps the X100net library for loading data into X100 servers directly. This interface is used by data loading tasks that work concurrently to load a set of data into a Vector table. Each data transfer task must follow the following flow:

• Use connect to connect to the X100 server handling this particular transfer. Note that the vector size used specifies the number of rows that can be sent per data block (called an IOB or I/O buffer within X100net).
• Call initializeTransfer to specify which columns are being loaded and which columns are excluded from the load. This call returns an array of column identifiers in the same order as the list of included column names. These identifiers are needed on calls to set data per column.
• For each vector size amount of data, invoke the set[Type] method for each included column to push data values into the IOB through the JNI layear. Direct buffers should be used to allow much more efficient movement of the data. Only non-null values should be pushed to the data buffer.
• Null markers consist of a byte value. The null marker is set to 1 to indicate a null value. A 0 value indicates not null. For null data values, the null marker should be enabled and no data value is pushed to the data buffer.
• Invoke the pushVector method to push the current vector of data values to the X100 server. If more data is available to load, use the data data[Type] set methods to push more data. Ensure to call pushBlock on the last, possibly incomplete set of data.
• Lastly, invoke close to send an end of data signal to the X100 server and to free up all resources on the JNI side. Once close is called, the connection cannot be used again.

Constructor Summary

Constructors

Constructor and Description
VectorDataTransfer()

Method Summary

Methods

Modifier and Type	Method and Description
void	close(int connectionID) Close the connection with the given connection ID.
int	connect(String host, String port, String user, String password, String table, int vectorSize) Connect to the X100 transfer agent at the given address.
int[]	initializeTransfer(int connectionID, String[] includedColumns, String[] excludedColumns) Initialize the transfer specifying the set of columns to load and the columns excluded from the load.
static boolean	isWindows() Verify if OS is windows
int	pushVector(int connectionID, int validRowCount) Push the vectors of data set using the set[Type] function(s) to the connected X100 server.
int	set128Bit(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set a vector of 16-byte integral values for the specified column.
int	setByte(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set a vector of 1-byte integral values for the specified column.
int	setDouble(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set a vector of 8-byte floating point values for the specified column.
int	setFloat(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set a vector of 4-byte floating point values for the specified column.
int	setInt(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set a vector of 4-byte values for the specified column.
int	setLong(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set a vector of 8-byte integral values for the specified column.
int	setShort(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set a vector of 2-byte integral values for the specified column.
int	setString(int connectionID, int columnIdx, ByteBuffer values, ByteBuffer nullMarkers) Set string data for the specified column.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VectorDataTransfer

public VectorDataTransfer()

Method Detail

isWindows

public static boolean isWindows()

Verify if OS is windows

Returns:

true if windows

connect

public int connect(String host,
          String port,
          String user,
          String password,
          String table,
          int vectorSize)
            throws IllegalArgumentException,
                   IOException

Connect to the X100 transfer agent at the given address. The identifier of the connection is returned. The identifier is required on other API calls.

Parameters:

host - name of the Vector server handling this transfer

port - IP port of the transfer connection

user - data transfer user name (generated by Vector)

password - data transfer password (generated by Vector)

table - name of the target table

vectorSize - cardinality of vectors to use for sending data (defaults to 1024)

Returns:

connection identifier or -1 if the call failed.

Throws:

IllegalArgumentException - thrown if any of the given arguments are invalid (i.e. null)

IOException - thrown for errors interfacing with X100

initializeTransfer

public int[] initializeTransfer(int connectionID,
                       String[] includedColumns,
                       String[] excludedColumns)
                         throws IllegalArgumentException,
                                IOException

Initialize the transfer specifying the set of columns to load and the columns excluded from the load. An array of column identifiers are returned, one per given included column name in position order. The column identifier is used when setting data values for a column (for example: setDouble).

Parameters:

connectionID - connection identifier

includedColumns - list of the names of columns to load (should never be empty)

excludedColumns - list of the names of columns not being loaded (may be empty)

Returns:

list of column identifiers in the same order as the given included column names

Throws:

IllegalArgumentException

IOException

setByte

public int setByte(int connectionID,
          int columnIdx,
          ByteBuffer values,
          ByteBuffer nullMarkers)
            throws IOException

Set a vector of 1-byte integral values for the specified column.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null data values

nullMarkers - buffer of null indicators (optional, null for no markers)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

setShort

public int setShort(int connectionID,
           int columnIdx,
           ByteBuffer values,
           ByteBuffer nullMarkers)
             throws IOException

Set a vector of 2-byte integral values for the specified column.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null data values

nullMarkers - buffer of null indicators (optional, null for no markers)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

setInt

public int setInt(int connectionID,
         int columnIdx,
         ByteBuffer values,
         ByteBuffer nullMarkers)
           throws IOException

Set a vector of 4-byte values for the specified column.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null data values

nullMarkers - buffer of null indicators (optional, null for no markers)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

setLong

public int setLong(int connectionID,
          int columnIdx,
          ByteBuffer values,
          ByteBuffer nullMarkers)
            throws IOException

Set a vector of 8-byte integral values for the specified column.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null data values

nullMarkers - buffer of null indicators (optional, null for no markers)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

set128Bit

public int set128Bit(int connectionID,
            int columnIdx,
            ByteBuffer values,
            ByteBuffer nullMarkers)
              throws IOException

Set a vector of 16-byte integral values for the specified column.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null data values

nullMarkers - buffer of null indicators (optional, null for no markers)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

setDouble

public int setDouble(int connectionID,
            int columnIdx,
            ByteBuffer values,
            ByteBuffer nullMarkers)
              throws IOException

Set a vector of 8-byte floating point values for the specified column.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null data values

nullMarkers - buffer of null indicators (optional, null for no markers)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

setFloat

public int setFloat(int connectionID,
           int columnIdx,
           ByteBuffer values,
           ByteBuffer nullMarkers)
             throws IOException

Set a vector of 4-byte floating point values for the specified column.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null data values

nullMarkers - buffer of null indicators (optional, null for no markers)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

setString

public int setString(int connectionID,
            int columnIdx,
            ByteBuffer values,
            ByteBuffer nullMarkers)
              throws IOException

Set string data for the specified column. Each string value must be terminated with a null character to mark the end of the string. The string data must not contain a zero byte since a zero byte is used to delineate the end of a string.

Parameters:

connectionID - connection identifier

columnIdx - identifier of the column to load

values - buffer of non-null string data values

nullMarkers - buffer of null indicators (optional)

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O errors that may occur

pushVector

public int pushVector(int connectionID,
             int validRowCount)
               throws IOException

Push the vectors of data set using the set[Type] function(s) to the connected X100 server. Specify the number of rows with valid data within the current vector buffer. The row count should be constrained by the vector size specified during creation of the connection. Each column should have pushed the same number of values.

Parameters:

connectionID - connection identifier

validRowCount - the number of rows with valid data to push to X100

Returns:

0 for success; -1 for failure

Throws:

IOException - thrown for any I/O error sending the data to X100

close

public void close(int connectionID)
           throws IOException

Close the connection with the given connection ID.

Parameters:

connectionID - identifier of the connection

Throws:

IOException