Class NemoPlanRewriter

  • All Implemented Interfaces:

    public final class NemoPlanRewriter
    extends java.lang.Object
    implements PlanRewriter
    Rewrites the physical plan during execution, to enforce the optimizations of Nemo RunTimePasses.

    A high-level flow of a rewrite is as follows: Runtime - (PhysicalPlan-level info) - NemoPlanRewriter - (IRDAG-level info) - NemoOptimizer - (new IRDAG) - NemoPlanRewriter - (new PhysicalPlan) - Runtime

    Here, the NemoPlanRewriter acts as a translator between the Runtime that only understands PhysicalPlan-level info, and the NemoOptimizer that only understands IRDAG-level info.

    This decoupling between the NemoOptimizer and the Runtime lets Nemo optimization policies dynamically control distributed execution behaviors, and at the same time enjoy correctness/reusability/composability properties that the IRDAG abstraction provides.

    • Method Detail

      • setCurrentIRDAG

        public void setCurrentIRDAG​(IRDAG currentIRDAG)
      • setCurrentPhysicalPlan

        public void setCurrentPhysicalPlan​(PhysicalPlan currentPhysicalPlan)
      • rewrite

        public PhysicalPlan rewrite​(int messageId)
        Specified by:
        rewrite in interface PlanRewriter
        messageId - of the rewrite.
        physical plan.
      • accumulate

        public void accumulate​(int messageId,
                               java.util.Set<StageEdge> targetEdges,
                               java.lang.Object data)
        Accumulate the data needed in Plan Rewrite. DATA_NOT_AUGMENTED indicates that the information need in rewrite is not stored in RunTimePassMessageEntry, and we should explicitly generate it using Prophet class. In this case, the data will contain only one entry with key as DATA_NOT_AUGMENTED.
        Specified by:
        accumulate in interface PlanRewriter
        messageId - of the rewrite.
        targetEdges - edges to change during rewrite.
        data - to accumulate.