Class NemoPlanRewriter
- java.lang.Object
-
- org.apache.nemo.compiler.backend.nemo.NemoPlanRewriter
-
- All Implemented Interfaces:
PlanRewriter
public final class NemoPlanRewriter extends java.lang.Object implements PlanRewriter
Rewrites the physical plan during execution, to enforce the optimizations of Nemo RunTimePasses.A high-level flow of a rewrite is as follows: Runtime - (PhysicalPlan-level info) - NemoPlanRewriter - (IRDAG-level info) - NemoOptimizer - (new IRDAG) - NemoPlanRewriter - (new PhysicalPlan) - Runtime
Here, the NemoPlanRewriter acts as a translator between the Runtime that only understands PhysicalPlan-level info, and the NemoOptimizer that only understands IRDAG-level info.
This decoupling between the NemoOptimizer and the Runtime lets Nemo optimization policies dynamically control distributed execution behaviors, and at the same time enjoy correctness/reusability/composability properties that the IRDAG abstraction provides.
-
-
Constructor Summary
Constructors Constructor Description NemoPlanRewriter(NemoOptimizer nemoOptimizer, NemoBackend nemoBackend, org.apache.reef.tang.InjectionFuture<SimulationScheduler> simulationSchedulerInjectionFuture, PhysicalPlanGenerator physicalPlanGenerator)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description void
accumulate(int messageId, java.util.Set<StageEdge> targetEdges, java.lang.Object data)
Accumulate the data needed in Plan Rewrite.PhysicalPlan
rewrite(int messageId)
void
setCurrentIRDAG(IRDAG currentIRDAG)
void
setCurrentPhysicalPlan(PhysicalPlan currentPhysicalPlan)
-
-
-
Constructor Detail
-
NemoPlanRewriter
@Inject public NemoPlanRewriter(NemoOptimizer nemoOptimizer, NemoBackend nemoBackend, org.apache.reef.tang.InjectionFuture<SimulationScheduler> simulationSchedulerInjectionFuture, PhysicalPlanGenerator physicalPlanGenerator)
-
-
Method Detail
-
setCurrentIRDAG
public void setCurrentIRDAG(IRDAG currentIRDAG)
-
setCurrentPhysicalPlan
public void setCurrentPhysicalPlan(PhysicalPlan currentPhysicalPlan)
-
rewrite
public PhysicalPlan rewrite(int messageId)
- Specified by:
rewrite
in interfacePlanRewriter
- Parameters:
messageId
- of the rewrite.- Returns:
- physical plan.
-
accumulate
public void accumulate(int messageId, java.util.Set<StageEdge> targetEdges, java.lang.Object data)
Accumulate the data needed in Plan Rewrite. DATA_NOT_AUGMENTED indicates that the information need in rewrite is not stored in RunTimePassMessageEntry, and we should explicitly generate it using Prophet class. In this case, the data will contain only one entry with key as DATA_NOT_AUGMENTED.- Specified by:
accumulate
in interfacePlanRewriter
- Parameters:
messageId
- of the rewrite.targetEdges
- edges to change during rewrite.data
- to accumulate.
-
-