Class SourceVertex<O>

    • Constructor Detail

      • SourceVertex

        public SourceVertex()
        Constructor for SourceVertex.
      • SourceVertex

        public SourceVertex​(SourceVertex that)
        Copy Constructor for SourceVertex.
        that - the source object for copying
    • Method Detail

      • isBounded

        public abstract boolean isBounded()
        true if it is bounded source
      • getReadables

        public abstract java.util.List<Readable<O>> getReadables​(int desiredNumOfSplits)
                                                          throws java.lang.Exception
        Gets parallel readables.
        desiredNumOfSplits - number of splits desired.
        the list of readables.
        java.lang.Exception - if fail to get.
      • getEstimatedSizeBytes

        public abstract long getEstimatedSizeBytes()
        Gets the estimated size of bytes. Returns 0L if not applicable.
        size of input bytes.
      • clearInternalStates

        public abstract void clearInternalStates()
        Clears internal states, must be called after getReadables(). Concretely, this clears the huge list of input splits held by objects like BeamBoundedSourceVertex before sending the vertex to remote executors. Between clearing states of an existing vertex, and creating a new vertex, we've chosen the former approach to ensure consistent use of the same IRVertex object across the compiler, the master, and the executors.