Module datarush.hadoop.core
Package com.pervasive.datarush.hbase

Class ReadHBase

java.lang.Object
com.pervasive.datarush.operators.AbstractLogicalOperator
com.pervasive.datarush.operators.CompositeOperator
com.pervasive.datarush.hbase.KeyOperator
com.pervasive.datarush.hbase.KeyValueOperator
com.pervasive.datarush.hbase.ReadHBase
All Implemented Interfaces:
LogicalOperator
public class ReadHBase extends KeyValueOperator
Read a result set from HBase.

The default read behavior returns the latest version of any retrieved cells. The versionCount property controls the number of versions returned for each cell. Optional row key, qualifier key (when mapping family as sub table), and time key (when versionCount > 1) fields can be specified to uniquely id each DataRush record.

Cell versions can also be filtered by a time range. The startTime and endTime properties specify the time range.

Each partition will read its assigned regions in row key ascending, qualifier key ascending, time key descending order. If cells from multiple families are mapped then they will be joined in the following manor:

  • cells mapped via mapFamily() and mapFamilyRecord() will be joined using the row key, qualifier key, and time key fields.
  • The result of the first step will then be joined with cells mapped via mapCell() and mapCellRecord() using the row key, and time key fields.
See Also:

  • Constructor Details

    • ReadHBase

      public ReadHBase()

  • Method Details

    • getStartTime

      public Date getStartTime()
      Get time range filter start time
      Returns:
      start time.

    • getEndTime

      public Date getEndTime()
      Get time range filter end time
      Returns:
      end time.

    • setStartTime

      public void setStartTime(Date startTime)
      Set time range filter start time
      Parameters:
      startTime - time range start time inclusive.

    • setEndTime

      public void setEndTime(Date endTime)
      Set time range filter end time
      Parameters:
      endTime - time range end time exclusive

    • getVersionCount

      public long getVersionCount()
      Get cell version count
      Returns:
      the maximum number of versions to return for each cell.

    • setVersionCount

      public void setVersionCount(long versionCount)
      Set cell version count
      Parameters:
      versionCount - - the maximum number of versions to return for each cell. Defaults to 1.

    • getOmitHBaseScan

      public boolean getOmitHBaseScan()
      Get omit HBase Scan state
      Returns:
      boolean indicating whether to omit HBase Scan from result set.

    • setOmitHBaseScan

      public void setOmitHBaseScan(boolean omitHBaseScan)
      Set omit HBase Scan state. The result set will not include data in HBase memstore, instead the returned result set will consist of only data stored in HFiles and retrieved via HFile Reader exclusively.
      Parameters:
      omitHBaseScan - - boolean indicating whether to omit HBase Scan from result set. Defaults to false.

    • getOmitHFileReader

      public boolean getOmitHFileReader()
      Get omit HFile Reader state
      Returns:
      boolean indicating whether to omit HFile Reader from result set.

    • setOmitHFileReader

      public void setOmitHFileReader(boolean omitHFileReader)
      Set omit HFile Reader state. HFiles will not be accessed directly, instead the returned result set, consisting of both memstore and HFile data, will be retrieved via the HBase Scan api exclusively.
      Parameters:
      omitHFileReader - - boolean indicating whether to omit HFile Reader from result set. Defaults to false.

    • compose

      public void compose(CompositionContext ctx)
      Description copied from class: CompositeOperator
      Compose the body of this operator. Implementations should do the following:
      1. Perform any validation of configuration, input types, etc
      2. Instantiate and configure sub-operators, adding them to the provided context via the method OperatorComposable.add(O)
      3. Create necessary connections via the method OperatorComposable.connect(P, P). This includes connections from the composite's input ports to sub-operators, connections between sub-operators, and connections from sub-operators output ports to the composite's output ports
      Specified by:
      compose in class CompositeOperator
      Parameters:
      ctx - the context

    • getOutput

      public RecordPort getOutput()