Module datarush.library

Package com.pervasive.datarush.schema

Class PatternBasedDiscovery

java.lang.Object

com.pervasive.datarush.schema.PatternBasedDiscovery

All Implemented Interfaces:

TextRecordDiscoverer

Direct Known Subclasses:

JsonPatternBasedDiscovery

public class PatternBasedDiscovery
extends Object
implements TextRecordDiscoverer

Discovers a schema for delimited text by checking values against a mapping of regular expressions to data types.

Field Summary

Fields

Modifier and Type	Field	Description
static List<TypePattern>	DEFAULT_PATTERNS	The default patterns used in discovery when none are specified, in order: TypePattern.INT_PATTERN TypePattern.LONG_PATTERN TypePattern.DOUBLE_PATTERN TypePattern.MONEY_PATTERN TypePattern.DATE_PATTERN TypePattern.TIME_PATTERN TypePattern.TIMESTAMP_PATTERN TypePattern.BOOLEAN_PATTERN TypePattern.IP4ADDRESS_PATTERN TypePattern.IP6ADDRESS_PATTERN TypePattern.DURATION_PATTERN TypePattern.PERIOD_PATTERN

Constructor Summary

Constructors

Constructor	Description
PatternBasedDiscovery()	Creates a default mapping.
PatternBasedDiscovery(List<TypePattern> patterns)	Creates a mapping using the specified patterns.

Method Summary

Modifier and Type	Method	Description
TextRecord	discoverForRead(List<List<String>> rows, boolean containsHeader, TextConversionDefaults defaults)	Constructs a schema for reading based on the given file analysis.
TextRecord	generateForWrite(RecordTokenType type, TextConversionDefaults defaults)	Constructs a schema for writing based on the given record type.
protected TextDataType	mergeTypes(TextDataType l, TextDataType r)
protected TextDataType	predictType(String value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_PATTERNS

public static List<TypePattern> DEFAULT_PATTERNS

The default patterns used in discovery when none are specified, in order:

• TypePattern.INT_PATTERN
• TypePattern.LONG_PATTERN
• TypePattern.DOUBLE_PATTERN
• TypePattern.MONEY_PATTERN
• TypePattern.DATE_PATTERN
• TypePattern.TIME_PATTERN
• TypePattern.TIMESTAMP_PATTERN
• TypePattern.BOOLEAN_PATTERN
• TypePattern.IP4ADDRESS_PATTERN
• TypePattern.IP6ADDRESS_PATTERN
• TypePattern.DURATION_PATTERN
• TypePattern.PERIOD_PATTERN

Constructor Details

PatternBasedDiscovery

public PatternBasedDiscovery()

Creates a default mapping. The default mapping detects:

• TextTypes.JAVA_INT
• TextTypes.JAVA_LONG
• TextTypes.JAVA_DOUBLE
• TextTypes.JAVA_MONEY
• TextTypes.ISO_DATE
• TextTypes.ISO_TIMEOFDAY
• TextTypes.ISO_TIMESTAMP
• TextTypes.TRUE_FALSE
• TextTypes.TEXT_IPV4ADDRESS
• TextTypes.TEXT_IPV6ADDRESS
• TextTypes.JAVA_DURATION
• TextTypes.JAVA_PERIOD

See Also:

• DEFAULT_PATTERNS

PatternBasedDiscovery

public PatternBasedDiscovery(List<TypePattern> patterns)

Creates a mapping using the specified patterns. Patterns are tested in the order listed with the first matching pattern used to determine type. Conflicts between records on a field's type are resolved to TextTypes.STRING in most cases. If a field value matches the default null indicator, it is ignored for typing purposes.

Parameters:

patterns - the patterns against which to compare values. The list is in precedence order; the first pattern to match determines the type.

Method Details

predictType

protected TextDataType predictType(String value)

mergeTypes

protected TextDataType mergeTypes(TextDataType l,
 TextDataType r)

discoverForRead

public TextRecord discoverForRead(List<List<String>> rows,
 boolean containsHeader,
 TextConversionDefaults defaults)

Description copied from interface: TextRecordDiscoverer

Constructs a schema for reading based on the given file analysis. The analysis may or may not include a header row, as indicated.

Specified by:

discoverForRead in interface TextRecordDiscoverer

Parameters:

rows - the analyzed rows from a text file

containsHeader - indicates whether the first row of the analyzed file is a header

defaults - defaults for the discovered schema

Returns:

a schema appropriate for reading the file.

generateForWrite

public TextRecord generateForWrite(RecordTokenType type,
 TextConversionDefaults defaults)

Description copied from interface: TextRecordDiscoverer

Constructs a schema for writing based on the given record type.

Specified by:

generateForWrite in interface TextRecordDiscoverer

Parameters:

type - the record type of the data

defaults - defaults for the discovered schema

Returns:

a schema appropriate for writing data of the given type