Module datarush.hadoop.core

Package com.pervasive.datarush.hbase

Class HBasePartitioningFunction

java.lang.Object

com.pervasive.datarush.hbase.HBasePartitioningFunction

All Implemented Interfaces:

PartitioningFunction

public class HBasePartitioningFunction
extends Object
implements PartitioningFunction

A partitioning function that partitions records of data based on a HBase table.

Nested Class Summary

Nested classes/interfaces inherited from interface com.pervasive.datarush.ports.record.PartitioningFunction

PartitioningFunction.Evaluator

Constructor Summary

Constructors

Constructor	Description
HBasePartitioningFunction()
HBasePartitioningFunction(ModuleConfiguration moduleConfiguration, String tableName, HadoopConfiguration configuration, Set<String> families, List<String> keys)

Method Summary

Modifier and Type	Method	Description
HadoopConfiguration	getConfiguration()
PartitioningFunction.Evaluator	getEvaluator(PartitioningFunctionContext ctx)	Create an Evaluator bound to the specified context.
Set<String>	getFamilies()
List<String>	getKeys()
ModuleConfiguration	getModuleConfiguration()
String	getTableName()
boolean	isEquivalentPartitioning(PartitioningFunction other)	Returns true iff, given a single row of input, this function is guaranteed to produce the same partitioning as the other function.
boolean	isFunctionOfInput()	Returns true iff, given two identical rows of input, this function will return the same value.
static List<NodeAllocationPlan.AssignmentPlan>	localizeAssignments(NodeAllocationPlan nodeAlloc, List<String> serverNames, boolean balance)
PartitioningFunction	remapFieldReferences(Map<String,String> oldToNewMapping)	Creates a new function, equivalent to the given function, but with all field references renamed according to the given mapping.
void	setConfiguration(HadoopConfiguration configuration)
void	setFamilies(Set<String> families)
void	setKeys(List<String> keys)
void	setModuleConfiguration(ModuleConfiguration moduleConfiguration)
void	setTableName(String tableName)
void	validateForType(RecordTokenType type)	Performs validation of the specified input type

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HBasePartitioningFunction

public HBasePartitioningFunction(ModuleConfiguration moduleConfiguration,
                                 String tableName,
                                 HadoopConfiguration configuration,
                                 Set<String> families,
                                 List<String> keys)

HBasePartitioningFunction

public HBasePartitioningFunction()

Method Detail

getModuleConfiguration

public ModuleConfiguration getModuleConfiguration()

setModuleConfiguration

public void setModuleConfiguration(ModuleConfiguration moduleConfiguration)

getTableName

public String getTableName()

setTableName

public void setTableName(String tableName)

getConfiguration

public HadoopConfiguration getConfiguration()

setConfiguration

public void setConfiguration(HadoopConfiguration configuration)

getFamilies

public Set<String> getFamilies()

setFamilies

public void setFamilies(Set<String> families)

getKeys

public List<String> getKeys()

setKeys

public void setKeys(List<String> keys)

getEvaluator

public PartitioningFunction.Evaluator getEvaluator(PartitioningFunctionContext ctx)

Description copied from interface: PartitioningFunction

Create an Evaluator bound to the specified context.

Specified by:

getEvaluator in interface PartitioningFunction

Parameters:

ctx - the partitioning context

Returns:

an Evaluator bound to the specified context.

validateForType

public void validateForType(RecordTokenType type)

Description copied from interface: PartitioningFunction

Performs validation of the specified input type

Specified by:

validateForType in interface PartitioningFunction

Parameters:

type - the input type

isEquivalentPartitioning

public boolean isEquivalentPartitioning(PartitioningFunction other)

Description copied from interface: PartitioningFunction

Returns true iff, given a single row of input, this function is guaranteed to produce the same partitioning as the other function. A return of true thus implies the following:

both functions are deterministic both function are stateless (not sensitive to previous values read or to input ordering) In addition, in order to be consistent with PartitioningFunction.isFunctionOfInput(), this function must return false if PartitioningFunction.isFunctionOfInput() returns false.

Specified by:

isEquivalentPartitioning in interface PartitioningFunction

Parameters:

other - the other function

Returns:

whether this function produces the same results as the other function.

isFunctionOfInput

public boolean isFunctionOfInput()

Description copied from interface: PartitioningFunction

Returns true iff, given two identical rows of input, this function will return the same value. A return of true thus implies :

the function is deterministic the function is stateless (not sensitive to previous values read or to input ordering)

Specified by:

isFunctionOfInput in interface PartitioningFunction

Returns:

whether this function is a stateless, deterministic function of input

remapFieldReferences

public PartitioningFunction remapFieldReferences(Map<String,String> oldToNewMapping)

Description copied from interface: PartitioningFunction

Creates a new function, equivalent to the given function, but with all field references renamed according to the given mapping.

Specified by:

remapFieldReferences in interface PartitioningFunction

Parameters:

oldToNewMapping - a mapping from old names to new names. If a name is not present in the given mapping, it will remain as-is.

Returns:

a new function with all field reference renamed appropriately

localizeAssignments

public static List<NodeAllocationPlan.AssignmentPlan> localizeAssignments(NodeAllocationPlan nodeAlloc,
                                                                          List<String> serverNames,
                                                                          boolean balance)