Package org.apache.nemo.compiler.frontend.beam.source

Class BeamBoundedSourceVertex<O>

  • Type Parameters:
    O - output type.
    All Implemented Interfaces:
    java.io.Serializable, Cloneable<IRVertex>
    public final class BeamBoundedSourceVertex<O>
extends SourceVertex<org.apache.beam.sdk.util.WindowedValue<O>>
    SourceVertex implementation for BoundedSource.
    See Also:
    Serialized Form

    • Constructor Detail

      • BeamBoundedSourceVertex

        public BeamBoundedSourceVertex​(org.apache.beam.sdk.io.BoundedSource<O> source,
                               org.apache.beam.sdk.transforms.display.DisplayData displayData)
        Constructor of BeamBoundedSourceVertex.
        Parameters:
        source - BoundedSource to read from.
        displayData - data to display.

    • Method Detail

      • getClone

        public BeamBoundedSourceVertex getClone()
        Description copied from interface: Cloneable
        Creates and returns a copy of this object.

        The precise meaning of "copy" may depend on the class of the object. The general intent is that, all fields of the object are copied.

        Returns:
        a clone of this object.

      • isBounded

        public boolean isBounded()
        Specified by:
        isBounded in class SourceVertex<org.apache.beam.sdk.util.WindowedValue<O>>
        Returns:
        true if it is bounded source

      • getReadables

        public java.util.List<Readable<org.apache.beam.sdk.util.WindowedValue<O>>> getReadables​(int desiredNumOfSplits)
                                                                                 throws java.lang.Exception
        Description copied from class: SourceVertex
        Gets parallel readables.
        Specified by:
        getReadables in class SourceVertex<org.apache.beam.sdk.util.WindowedValue<O>>
        Parameters:
        desiredNumOfSplits - number of splits desired.
        Returns:
        the list of readables.
        Throws:
        java.lang.Exception - if fail to get.

      • getEstimatedSizeBytes

        public long getEstimatedSizeBytes()
        Description copied from class: SourceVertex
        Gets the estimated size of bytes. Returns 0L if not applicable.
        Specified by:
        getEstimatedSizeBytes in class SourceVertex<org.apache.beam.sdk.util.WindowedValue<O>>
        Returns:
        size of input bytes.

      • clearInternalStates

        public void clearInternalStates()
        Description copied from class: SourceVertex
        Clears internal states, must be called after getReadables(). Concretely, this clears the huge list of input splits held by objects like BeamBoundedSourceVertex before sending the vertex to remote executors. Between clearing states of an existing vertex, and creating a new vertex, we've chosen the former approach to ensure consistent use of the same IRVertex object across the compiler, the master, and the executors.
        Specified by:
        clearInternalStates in class SourceVertex<org.apache.beam.sdk.util.WindowedValue<O>>

      • getPropertiesAsJsonNode

        public com.fasterxml.jackson.databind.node.ObjectNode getPropertiesAsJsonNode()
        Overrides:
        getPropertiesAsJsonNode in class Vertex
        Returns:
        JSON representation of additional properties