java.lang.Object
- com.pervasive.datarush.operators.AbstractLogicalOperator
- - com.pervasive.datarush.operators.CompositeOperator
  - - com.pervasive.datarush.hbase.KeyOperator
    - - com.pervasive.datarush.hbase.KeyValueOperator

All Implemented Interfaces:

LogicalOperator

Direct Known Subclasses:

ReadHBase, WriteHBase
```
public abstract class KeyValueOperator
extends KeyOperator
```
Specifies data field mapping when accessing HBase.
A DataRush Field can be mapped to a select HBase cell or it can be mapped to all cells in a family as a sub table:
1. mapCell(java.lang.String, java.lang.String, java.lang.String) - Map a select cell within a column family as a field. Cells within a family can be of heterogeneous type and only the mapped cells are accessed. All mapped cell fields will be together in a single record. Optional row key, and time key fields can be specified to uniquely id each DataRush record.
2. mapFamily(java.lang.String, java.lang.String) - Map all cells within a column family as a sub table of fields. All cells within a family are of homogeneous type and all cells are accessed. Each cell within a family is contained in an individual record. Optional row key, qualifier key, and time key fields can be specified to uniquely id each DataRush record.
Mapping cells such that each cell contains a single field will serialize/deserialize the following DataRush data types using common HBase formats:
- TokenTypeConstant.INT - org.apache.hadoop.hbase.util.Bytes.toBytes(int)
- TokenTypeConstant.LONG - org.apache.hadoop.hbase.util.Bytes.toBytes(long)
- TokenTypeConstant.FLOAT - org.apache.hadoop.hbase.util.Bytes.toBytes(float)
- TokenTypeConstant.DOUBLE - org.apache.hadoop.hbase.util.Bytes.toBytes(double)
- TokenTypeConstant.NUMERIC - org.apache.hadoop.hbase.util.Bytes.toBytes(BigDecimal)
- TokenTypeConstant.STRING - org.apache.hadoop.hbase.util.Bytes.toBytes(String)
- TokenTypeConstant.BINARY - store/retrieve byte array as is
- All other data types will be serialized/deserialized using the default DataRush formats.
An HBase cell can also be mapped as a record of fields:
1. mapCellRecord(java.lang.String, java.lang.String, java.util.Map<java.lang.String, java.lang.String>) - Map a select cell within a column family as a record. Cells within a family can be of heterogeneous type and only the mapped cells are accessed. All mapped cell fields will be together in a single record.Optional row key, and time key fields can be specified to uniquely id each DataRush record.
2. mapFamilyRecord(java.lang.String, java.util.Map<java.lang.String, java.lang.String>) - Map all cells within a column family as a sub table of records. All cells within a family are of homogeneous type and all cells are accessed. Each cell within a family is contained in an individual record. Optional row key, qualifier key, and time key fields can be specified to uniquely id each DataRush record.
Mapping a single HBase cell as a record of fields will allow multiple fields to be packed into a single cell thus greatly increasing io performance at the expense of reduced version granularity. All fields packed together in a single cell are versioned together and therefore all fields must be present when writing. The default DataRush serialization is used exclusively in this case.
DataRush schema is persisted in HBase for any cell that DataRush writes to.
If a DataRush schema exists:
- DataRush read operations will return requested keys and/or fields as schema data type.
- DataRush write operations will check for type compatibility.
If a DataRush schema does not exist:
- a cell can only be mapped as a single field for reading.
- DataRush read operations will return the keys and/or fields as TokenTypeConstant.BINARY.
- DataRush write operations will create a schema upon first write.
See Also:

WriteHBase, ReadHBase, DeleteHBase

Field Summary
- Fields inherited from class com.pervasive.datarush.hbase.KeyOperator
  catalogTableName, cellSchemaFamilyName, keySchemaFamilyName, statsFamilyName

Constructor Summary

Constructors
Constructor Description

KeyValueOperator()

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method	Description
`Map<String,Map<String,Map<String,String>>>`	`getCellFieldMap()`	Get the HBase cell to field mapping.
`Map<String,Map<String,String>>`	`getFamilyFieldMap()`	Get the column family to field mapping.
`String`	`getHCatalogDatabase()`	Get the HCatalog database from which to retrieve a schema and mapping.
`List<String>`	`getHCatalogFields()`	Get the HCatalog fields to read or write.
`String`	`getHCatalogTable()`	Get the HCatalog table from which to retrieve a schema and mapping.
`void`	`mapCell(String familyName, String qualifier, String fieldName)`	Map Cell as a Field.
`void`	`mapCell(String familyName, String qualifier, List<String> schemaNames, List<String> fieldNames)`	Deprecated.
`void`	`mapCellRecord(String familyName, String qualifier, Map<String,String> fieldMap)`	Map Cell as a Record.
`void`	`mapFamily(String familyName, String fieldName)`	Map family as a sub-table of fields.
`void`	`mapFamily(String familyName, List<String> schemaNames, List<String> fieldNames)`	Deprecated.
`void`	`mapFamilyRecord(String familyName, Map<String,String> fieldMap)`	Map family as a sub-table of records.
`protected com.pervasive.datarush.hadoop.shims.hbase.TableSchema`	`mapFromHCatalog(MetadataContext ctx)`	Load a mapping from an existing HCatalog table.
`protected void`	`mapToHCatalog(MetadataContext ctx, RecordPort input)`	Write a mapping to a new HCatalog table.
`protected boolean`	`schemaSupportedByHCatalog(MetadataContext ctx)`	Determines if the current schema can be written to HCatalog.
`void`	`setCellFieldMap(Map<String,Map<String,Map<String,String>>> cellFieldMap)`	Set the HBase cell to field mapping.
`void`	`setFamilyFieldMap(Map<String,Map<String,String>> familyFieldMap)`	Set the column family to field mapping.
`void`	`setHCatalogDatabase(String database)`	Set the HCatalog database from which to retrieve a schema and mapping.
`void`	`setHCatalogFields(String... fields)`	Set the HCatalog fields to read or write.
`void`	`setHCatalogFields(List<String> fields)`	Set the HCatalog fields to read or write.
`void`	`setHCatalogTable(String table)`	Set the HCatalog table from which to retrieve a schema and mapping.
`protected boolean`	`tableExistsInHCatalog(MetadataContext ctx)`	Determines if the currently selected HCatalog table already exists.

Methods inherited from class com.pervasive.datarush.hbase.KeyOperator
addFamily, effectiveConfiguration, getConfiguration, getFamilies, getFilesystem, getHiveMetastore, getQualifierFieldMap, getRootDirectory, getRowFieldMap, getTableName, getTimeFieldName, getZookeeperParentZNode, getZookeeperPort, getZookeeperQuorum, mapQualifier, mapRow, mapRowRecord, setConfiguration, setFamilies, setFilesystem, setHiveMetastore, setQualifierFieldMap, setRootDirectory, setRowFieldMap, setTableName, setTimeFieldName, setZookeeperParentZNode, setZookeeperPort, setZookeeperQuorum

Methods inherited from class com.pervasive.datarush.operators.CompositeOperator
compose

Methods inherited from class com.pervasive.datarush.operators.AbstractLogicalOperator
disableParallelism, getInputPorts, getOutputPorts, newInput, newInput, newOutput, newRecordInput, newRecordInput, newRecordOutput, notifyError

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Constructor Detail
  - KeyValueOperator
```
public KeyValueOperator()
```
- Method Detail
  - getFamilyFieldMap
```
public Map<String,Map<String,String>> getFamilyFieldMap()
```
    Get the column family to field mapping.
    
    Returns:
    
    column family to field mapping
  - setFamilyFieldMap
```
public void setFamilyFieldMap(Map<String,Map<String,String>> familyFieldMap)
```
    Set the column family to field mapping.
    
    Parameters:
    
    familyFieldMap - column family field mapping
  - getCellFieldMap
```
public Map<String,Map<String,Map<String,String>>> getCellFieldMap()
```
    Get the HBase cell to field mapping.
    
    Returns:
    
    cell to field mapping
  - setCellFieldMap
```
public void setCellFieldMap(Map<String,Map<String,Map<String,String>>> cellFieldMap)
```
    Set the HBase cell to field mapping.
    
    Parameters:
    
    cellFieldMap - cell to field mapping
  - mapCellRecord
```
public void mapCellRecord(String familyName,
                          String qualifier,
                          Map<String,String> fieldMap)
```
    Map Cell as a Record. This method is used to map a select Cell as a Record. Any previous cell mapping for the specified family, qualifier is replaced.
    
    Parameters:
    
    familyName - the name of the HBase column family.
    
    qualifier - the HBase column qualifier identifying the cell to be mapped as a Record.
    
    fieldMap - cell record schema name to DataRush field name map.
  - mapCell
```
public void mapCell(String familyName,
                    String qualifier,
                    String fieldName)
```
    Map Cell as a Field. This method is used to map a select cell as a field. Any previous cell mapping for the specified family, qualifier is replaced.
    
    Parameters:
    
    familyName - the name of the HBase column family.
    
    qualifier - the HBase column qualifier identifying the cell to be mapped as a Field.
    
    fieldName - the field name to be mapped to the specified cell qualifier.
  - mapCell
```
@Deprecated
public void mapCell(String familyName,
                    String qualifier,
                    List<String> schemaNames,
                    List<String> fieldNames)
```
    Deprecated.
  - mapFamilyRecord
```
public void mapFamilyRecord(String familyName,
                            Map<String,String> fieldMap)
```
    Map family as a sub-table of records. This method is used to map all cells within a family as a sub-table of records. Any previous mapping for the specified family is replaced.
    
    Parameters:
    
    familyName - the name of the HBase column family.
    
    fieldMap - cell record schema name to DataRush field name map.
  - mapFamily
```
public void mapFamily(String familyName,
                      String fieldName)
```
    Map family as a sub-table of fields. This method is used to map all cells within a family as a sub-table of fields. Any previous mapping for the specified family is replaced.
    
    Parameters:
    
    familyName - the name of the HBase column family.
    
    fieldName - the field name to be mapped to the specified family.
  - mapFamily
```
@Deprecated
public void mapFamily(String familyName,
                      List<String> schemaNames,
                      List<String> fieldNames)
```
    Deprecated.
  - getHCatalogDatabase
```
public String getHCatalogDatabase()
```
    Get the HCatalog database from which to retrieve a schema and mapping.
    
    Returns:
    
    the HCatalog database
  - setHCatalogDatabase
```
public void setHCatalogDatabase(String database)
```
    Set the HCatalog database from which to retrieve a schema and mapping.
    
    Parameters:
    
    database - the HCatalog database
  - getHCatalogTable
```
public String getHCatalogTable()
```
    Get the HCatalog table from which to retrieve a schema and mapping.
    
    Returns:
    
    the HCatalog table
  - setHCatalogTable
```
public void setHCatalogTable(String table)
```
    Set the HCatalog table from which to retrieve a schema and mapping.
    
    Parameters:
    
    table - the HCatalog table
  - getHCatalogFields
```
public List<String> getHCatalogFields()
```
    Get the HCatalog fields to read or write.
    
    Returns:
    
    the a list of field names, or null if not set
  - setHCatalogFields
```
public void setHCatalogFields(List<String> fields)
```
    Set the HCatalog fields to read or write. If not set, all defined fields will be read or written.
    
    Parameters:
    
    fields - a list of field names
  - setHCatalogFields
```
public void setHCatalogFields(String... fields)
```
    Set the HCatalog fields to read or write. If not set, all defined fields will be read or written.
    
    Parameters:
    
    fields - a list of field names
  - mapFromHCatalog
```
protected com.pervasive.datarush.hadoop.shims.hbase.TableSchema mapFromHCatalog(MetadataContext ctx)
```
    Load a mapping from an existing HCatalog table.
    
    Parameters:
    
    ctx -
  - mapToHCatalog
```
protected void mapToHCatalog(MetadataContext ctx,
                             RecordPort input)
```
    Write a mapping to a new HCatalog table.
    
    Parameters:
    
    ctx -
  - tableExistsInHCatalog
```
protected boolean tableExistsInHCatalog(MetadataContext ctx)
```
    Determines if the currently selected HCatalog table already exists.
    
    Parameters:
    
    ctx -
    
    Returns:
    
    true if the table exists, false otherwise
  - schemaSupportedByHCatalog
```
protected boolean schemaSupportedByHCatalog(MetadataContext ctx)
```
    Determines if the current schema can be written to HCatalog.
    A schema is supported by HCatalog if its only mappings are cells to fields.
    
    Parameters:
    
    ctx -
    
    Returns:
    
    true

Class KeyValueOperator

Field Summary

Fields inherited from class com.pervasive.datarush.hbase.KeyOperator

Constructor Summary

Method Summary

Methods inherited from class com.pervasive.datarush.hbase.KeyOperator

Methods inherited from class com.pervasive.datarush.operators.CompositeOperator

Methods inherited from class com.pervasive.datarush.operators.AbstractLogicalOperator

Methods inherited from class java.lang.Object

Constructor Detail

KeyValueOperator

Method Detail

getFamilyFieldMap

setFamilyFieldMap

getCellFieldMap

setCellFieldMap

mapCellRecord

mapCell

mapCell

mapFamilyRecord

mapFamily

mapFamily

getHCatalogDatabase

setHCatalogDatabase

getHCatalogTable

setHCatalogTable

getHCatalogFields

setHCatalogFields

setHCatalogFields

mapFromHCatalog

mapToHCatalog

tableExistsInHCatalog

schemaSupportedByHCatalog