Class SkewRunTimePass
- java.lang.Object
-
- org.apache.nemo.common.pass.Pass
-
- org.apache.nemo.compiler.optimizer.pass.runtime.RunTimePass<Map<Object,Long>>
-
- org.apache.nemo.compiler.optimizer.pass.runtime.SkewRunTimePass
-
public final class SkewRunTimePass extends RunTimePass<Map<Object,Long>>
Dynamic optimization pass for handling data skew. Using a map of key to partition size as a metric used for dynamic optimization, this RunTimePass identifies a number of keys with big partition sizes(skewed key) and evenly redistributes data via overwriting incoming edges of destination tasks.
-
-
Constructor Summary
Constructors Constructor Description SkewRunTimePass()
SkewRunTimePass(int numOfSkewedKeys)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description IRDAG
apply(IRDAG irdag, Message<Map<Object,Long>> message)
-
Methods inherited from class org.apache.nemo.common.pass.Pass
addCondition, getCondition
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface java.util.function.BiFunction
andThen
-
-