Module datarush.library
Package com.pervasive.datarush.ports

Class PortSettings

  • public final class PortSettings
extends Object
    Utility class for working with port settings. It provides a number of factory methods for creating new PortSetting objects.

    • Method Detail

      • apply

        public static EngineConfig apply​(EngineConfig config,
                                 PortSetting... settings)
        Specifies the default properties for ports in the graph. Unspecified port settings will assume the value of the configuration to which the setting is applied.
        Parameters:
        config - the original EngineConfig
        settings - the desired port settings
        Returns:
        a new EngineConfig with the settings modified

      • immediate

        public static PortSetting immediate()
        Specifies that the port should immediately publish data when pushed. This is equivalent to batched(1).

        This option should only be used on a port-by-port basis and not globally.

        Returns:
        a port setting specifying a batch size of one
        See Also:
        batchSize(int)

      • batchSize

        public static PortSetting batchSize​(int size)
        Specifies the port should publish pushed data in batches of the specified size. Data will not be published to readers until a full batch is ready, end of data is pushed, or the flush() method is called on the port.

        Batching reduces overhead due to synchronization between dataflow processes, as they need to synchronize less frequently. However, large batch sizes can also be detrimental as it increases both memory usage and latency. Batch size should only be modified with care.

        Parameters:
        size - the batch size
        Returns:
        a port setting defining the batch size to use

      • writeahead

        public static PortSetting writeahead​(int size)
        Specifies the number of unread batches which a port can publish before blocking. A batch is considered unread if there exists any reader dataflow process which has not read it.

        Writeahead allows variation in writer performance to be smoothed out by buffering "fast" writers. Increasing this value can also decrease contention between readers, as access is spread across more batches. However, this also increases memory usage.

        Parameters:
        size - the number of batches
        Returns:
        a port setting defining the writeahead limit

      • sizeByReaders

        public static PortSetting sizeByReaders​(boolean enabled)
        Specifies whether the initial writeahead for a port should be automatically determined based on the number of readers. Even if enabled, the writeahead limit will be at least the value specified; the limit is only increased for queues having more readers than the defined limit.

        Enabling this can help reduce contention on queues having a large number of readers.

        Parameters:
        enabled - whether to enable
        Returns:
        a port setting declaring that ports should adjust writeahead limits upwards based on the number of readers
        See Also:
        writeahead(int)

      • spoolThreshold

        public static PortSetting spoolThreshold​(int size)
        Specifies the threshold at which the writer begins writing published batches to disk. After the threshold is reached, the writer never blocks due to unread batches. Batches are only written to disk if the (possibly increased) writeahead limit will be exceeded.

        This setting can be used to place an upper bound on the memory used for data queues between dataflow processes. Performance will possibly degrade, but OutOfMemoryErrors which kill the graph will be avoided.

        Parameters:
        size - the number of unread batches at which overflow spooling behavior is triggered
        Returns:
        a port setting
        See Also:
        writeahead(int)

      • noSpooling

        public static PortSetting noSpooling()
        Disables writers writing published batches to disk. All batches are kept in memory. This will increase memory usage, possibly causing OutOfMemoryErrors when queue expansion is occurring, but avoids the performance overhead of disk I/O when memory buffers are full.
        Returns:
        a port setting
        See Also:
        spoolThreshold(int)