com.pervasive.datarush.ports

Class PortSettings

java.lang.Object

com.pervasive.datarush.ports.PortSettings

public final class PortSettings
extends Object

Utility class for working with port settings. It provides a number of factory methods for creating new PortSetting objects.

Method Summary

Methods

Modifier and Type	Method and Description
static EngineConfig	apply(EngineConfig config, PortSetting... settings) Specifies the default properties for ports in the graph.
static PortSetting	batchSize(int size) Specifies the port should publish pushed data in batches of the specified size.
static PortSetting	immediate() Specifies that the port should immediately publish data when pushed.
static PortSetting	noSpooling() Disables writers writing published batches to disk.
static PortSetting	sizeByReaders(boolean enabled) Specifies whether the initial writeahead for a port should be automatically determined based on the number of readers.
static PortSetting	spoolThreshold(int size) Specifies the threshold at which the writer begins writing published batches to disk.
static PortSetting	writeahead(int size) Specifies the number of unread batches which a port can publish before blocking.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

apply

public static EngineConfig apply(EngineConfig config,
                 PortSetting... settings)

Specifies the default properties for ports in the graph. Unspecified port settings will assume the value of the configuration to which the setting is applied.

Parameters:

config - the original EngineConfig

settings - the desired port settings

Returns:

a new EngineConfig with the settings modified

immediate

public static PortSetting immediate()

Specifies that the port should immediately publish data when pushed. This is equivalent to batched(1).

This option should only be used on a port-by-port basis and not globally.

Returns:

a port setting specifying a batch size of one

See Also:

batchSize(int)

batchSize

public static PortSetting batchSize(int size)

Specifies the port should publish pushed data in batches of the specified size. Data will not be published to readers until a full batch is ready, end of data is pushed, or the flush() method is called on the port.

Batching reduces overhead due to synchronization between dataflow processes, as they need to synchronize less frequently. However, large batch sizes can also be detrimental as it increases both memory usage and latency. Batch size should only be modified with care.

Parameters:

size - the batch size

Returns:

a port setting defining the batch size to use

writeahead

public static PortSetting writeahead(int size)

Specifies the number of unread batches which a port can publish before blocking. A batch is considered unread if there exists any reader dataflow process which has not read it.

Writeahead allows variation in writer performance to be smoothed out by buffering "fast" writers. Increasing this value can also decrease contention between readers, as access is spread across more batches. However, this also increases memory usage.

Parameters:

size - the number of batches

Returns:

a port setting defining the writeahead limit

sizeByReaders

public static PortSetting sizeByReaders(boolean enabled)

Specifies whether the initial writeahead for a port should be automatically determined based on the number of readers. Even if enabled, the writeahead limit will be at least the value specified; the limit is only increased for queues having more readers than the defined limit.

Enabling this can help reduce contention on queues having a large number of readers.

Parameters:

enabled - whether to enable

Returns:

a port setting declaring that ports should adjust writeahead limits upwards based on the number of readers

See Also:

writeahead(int)

spoolThreshold

public static PortSetting spoolThreshold(int size)

Specifies the threshold at which the writer begins writing published batches to disk. After the threshold is reached, the writer never blocks due to unread batches. Batches are only written to disk if the (possibly increased) writeahead limit will be exceeded.

This setting can be used to place an upper bound on the memory used for data queues between dataflow processes. Performance will possibly degrade, but OutOfMemoryErrors which kill the graph will be avoided.

Parameters:

size - the number of unread batches at which overflow spooling behavior is triggered

Returns:

a port setting

See Also:

writeahead(int)

noSpooling

public static PortSetting noSpooling()

Disables writers writing published batches to disk. All batches are kept in memory. This will increase memory usage, possibly causing OutOfMemoryErrors when queue expansion is occurring, but avoids the performance overhead of disk I/O when memory buffers are full.

Returns:

a port setting

See Also:

spoolThreshold(int)