Class RecordMetadata

  • All Implemented Interfaces:
    PortMetadata

    public final class RecordMetadata
    extends Object
    implements PortMetadata
    The metadata associated with record ports. Operator authors should generally not need to interact with this class directly since there are a number of convenience methods on RecordPort for creating, modifying, and examining record metadata.
    • Constructor Detail

      • RecordMetadata

        public RecordMetadata​(RecordTokenType type)
        Create a RecordMetadata of the given type and unspecified ordering and partitioning.
        Parameters:
        type - the record type
      • RecordMetadata

        public RecordMetadata​(RecordTokenType type,
                              DataOrdering dataOrdering,
                              DataDistribution dataDistribution)
        Create a RecordMetadata of the given type, ordering, and distribution.
        Parameters:
        type - the record type
        dataOrdering - the data ordering
        dataDistribution - the data distribution
    • Method Detail

      • getType

        public RecordTokenType getType()
        Returns the type of the record.
        Returns:
        the type of the record.
      • getDataOrdering

        public DataOrdering getDataOrdering()
        Returns the data ordering of the record port.
        Returns:
        the data ordering of the record port.
      • getDataDistribution

        public DataDistribution getDataDistribution()
        Returns the data partitioning of the record port.
        Returns:
        the data partitioning of the record port.
      • getStagingOptions

        public DatasetOptions getStagingOptions()
        Returns the staging options of the record port.
        Returns:
        the staging options of the record port.
      • ordered

        public RecordMetadata ordered​(DataOrdering dataOrdering)
        Returns a new RecordMetadata, equivalent to this, but with ordering set to the specified value.
        Parameters:
        dataOrdering - the new ordering
        Returns:
        a new RecordMetadata with ordering adjusted
      • distributed

        public RecordMetadata distributed​(DataDistribution dataDistribution)
        Returns a new RecordMetadata, equivalent to this, but with dataDistribution set to the specified value.
        Parameters:
        dataDistribution - the new distribution
        Returns:
        a new RecordMetadata with data distribution adjusted
      • stagingOptions

        public RecordMetadata stagingOptions​(DatasetOptions stagingOptions)
        Returns a new RecordMetadata, equivalent to this, but with stagingOptions set to the specified value.
        Parameters:
        stagingOptions - the new staging options.
        Returns:
        a new RecordMetadata with staging options adjusted
      • remap

        public RecordMetadata remap​(FieldRemapping mapping)
        Computes the resulting metadata after applying the specified schema remapping. Data ordering and distribution are modified as appropriate to fit the new schema. Staging options are not preserved.

        This method assumes the values of fields are not being modified and that the relative ordering of records in the flow is unchanged.

        Parameters:
        mapping - the remapping to apply to the schema
        Returns:
        the metadata after the remapping
      • mutating

        public RecordMetadata mutating​(List<String> modifiedFields)
        Computes the resulting metadata assuming that the specified fields may be mutated. Data ordering and distribution are modified as appropriate given the possibility of change to field values. Staging options are not preserved.

        This methods assumes that both the underlying schema and the relative ordering of records in the flow is unchanged.

        Parameters:
        modifiedFields - the fields that may be modified
        Returns:
        the metadata after changing field values
      • getPortFactory

        public LogicalPortFactory<?> getPortFactory()
        Description copied from interface: PortMetadata
        Public only as a matter of implementation; not intended for external use. Returns a port factory that can create ports of the type associated with this metadata.
        Specified by:
        getPortFactory in interface PortMetadata
        Returns:
        a port factory