Package org.apache.nemo.compiler.backend.nemo.prophet

Class ParallelismProphet

  • All Implemented Interfaces:
    Prophet<java.lang.String,​java.lang.Long>
    public final class ParallelismProphet
extends java.lang.Object
implements Prophet<java.lang.String,​java.lang.Long>
    A prophet for Parallelism.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.util.Map<java.lang.String,​java.lang.Long> calculate()
      Launch SimulationScheduler and find out the optimal parallelism.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Constructor Detail

      • ParallelismProphet

        public ParallelismProphet​(IRDAG irdag,
                          PhysicalPlan physicalPlan,
                          SimulationScheduler simulationScheduler,
                          PhysicalPlanGenerator physicalPlanGenerator,
                          java.util.Set<StageEdge> edgesToOptimize)
        Default constructor for ParallelismProphet.
        Parameters:
        irdag - current IRDAG
        physicalPlan - current PhysicalPlan
        simulationScheduler - SimulationScheduler to launch
        physicalPlanGenerator - PhysicalPlanGenerator to make physical plan which will be launched by simulation scheduler
        edgesToOptimize - edges to optimize at runtime pass

    • Method Detail

      • calculate

        public java.util.Map<java.lang.String,​java.lang.Long> calculate()
        Launch SimulationScheduler and find out the optimal parallelism. For now, the number of candidate parallelisms is seven, so we iterate seven times. In each iteration index i, the candidate parallelism is calculated by dividing the i-th power of two from partitonerProperty (which is guaranteed to be one of 1024, 2048, 4096. For more information, please refer to SamplingTaskSizingPass). This approach is taken to guarantee the equal length of each partition, which will be updated in DynamicTaskSizingRuntimePass.
        Specified by:
        calculate in interface Prophet<java.lang.String,​java.lang.Long>
        Returns:
        Map of one element, with key "opt.parallelism".