Class SkewRunTimePass

  • All Implemented Interfaces:
    java.util.function.BiFunction<IRDAG,​Message<java.util.Map<java.lang.Object,​java.lang.Long>>,​IRDAG>

    public final class SkewRunTimePass
    extends RunTimePass<java.util.Map<java.lang.Object,​java.lang.Long>>
    Dynamic optimization pass for handling data skew. Using a map of key to partition size as a metric used for dynamic optimization, this RunTimePass identifies a number of keys with big partition sizes(skewed key) and evenly redistributes data via overwriting incoming edges of destination tasks.
    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      IRDAG apply​(IRDAG irdag, Message<java.util.Map<java.lang.Object,​java.lang.Long>> message)  
      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
      • Methods inherited from interface java.util.function.BiFunction

        andThen
    • Constructor Detail

      • SkewRunTimePass

        public SkewRunTimePass()
      • SkewRunTimePass

        public SkewRunTimePass​(int numOfSkewedKeys)
    • Method Detail

      • apply

        public IRDAG apply​(IRDAG irdag,
                           Message<java.util.Map<java.lang.Object,​java.lang.Long>> message)