Package org.apache.nemo.compiler.backend.nemo

Class NemoPlanRewriter

java.lang.Object

org.apache.nemo.compiler.backend.nemo.NemoPlanRewriter

All Implemented Interfaces:

PlanRewriter

public final class NemoPlanRewriter
extends java.lang.Object
implements PlanRewriter

Rewrites the physical plan during execution, to enforce the optimizations of Nemo RunTimePasses.

A high-level flow of a rewrite is as follows: Runtime - (PhysicalPlan-level info) - NemoPlanRewriter - (IRDAG-level info) - NemoOptimizer - (new IRDAG) - NemoPlanRewriter - (new PhysicalPlan) - Runtime

Here, the NemoPlanRewriter acts as a translator between the Runtime that only understands PhysicalPlan-level info, and the NemoOptimizer that only understands IRDAG-level info.

This decoupling between the NemoOptimizer and the Runtime lets Nemo optimization policies dynamically control distributed execution behaviors, and at the same time enjoy correctness/reusability/composability properties that the IRDAG abstraction provides.

Constructor Summary

Constructors

Constructor	Description
NemoPlanRewriter(NemoOptimizer nemoOptimizer, NemoBackend nemoBackend, org.apache.reef.tang.InjectionFuture<SimulationScheduler> simulationSchedulerInjectionFuture, PhysicalPlanGenerator physicalPlanGenerator)

Method Summary

Modifier and Type	Method	Description
void	accumulate(int messageId, java.util.Set<StageEdge> targetEdges, java.lang.Object data)	Accumulate the data needed in Plan Rewrite.
PhysicalPlan	rewrite(int messageId)
void	setCurrentIRDAG(IRDAG currentIRDAG)
void	setCurrentPhysicalPlan(PhysicalPlan currentPhysicalPlan)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NemoPlanRewriter

@Inject
public NemoPlanRewriter(NemoOptimizer nemoOptimizer,
                        NemoBackend nemoBackend,
                        org.apache.reef.tang.InjectionFuture<SimulationScheduler> simulationSchedulerInjectionFuture,
                        PhysicalPlanGenerator physicalPlanGenerator)

Method Detail

setCurrentIRDAG

public void setCurrentIRDAG(IRDAG currentIRDAG)

setCurrentPhysicalPlan

public void setCurrentPhysicalPlan(PhysicalPlan currentPhysicalPlan)

rewrite

public PhysicalPlan rewrite(int messageId)

Specified by:

rewrite in interface PlanRewriter

Parameters:

messageId - of the rewrite.

Returns:

physical plan.

accumulate

public void accumulate(int messageId,
                       java.util.Set<StageEdge> targetEdges,
                       java.lang.Object data)

Accumulate the data needed in Plan Rewrite. DATA_NOT_AUGMENTED indicates that the information need in rewrite is not stored in RunTimePassMessageEntry, and we should explicitly generate it using Prophet class. In this case, the data will contain only one entry with key as DATA_NOT_AUGMENTED.

Specified by:

accumulate in interface PlanRewriter

Parameters:

messageId - of the rewrite.

targetEdges - edges to change during rewrite.

data - to accumulate.