Uses of Class
org.apache.nemo.common.ir.IRDAG

Packages that use IRDAG

Package	Description
org.apache.nemo.client
org.apache.nemo.common.ir
org.apache.nemo.common.pass
org.apache.nemo.common.test
org.apache.nemo.compiler
org.apache.nemo.compiler.backend
org.apache.nemo.compiler.backend.nemo
org.apache.nemo.compiler.backend.nemo.prophet
org.apache.nemo.compiler.frontend.beam
org.apache.nemo.compiler.optimizer
org.apache.nemo.compiler.optimizer.pass.compiletime.annotating
org.apache.nemo.compiler.optimizer.pass.compiletime.composite
org.apache.nemo.compiler.optimizer.pass.compiletime.reshaping
org.apache.nemo.compiler.optimizer.pass.runtime
org.apache.nemo.compiler.optimizer.policy
org.apache.nemo.runtime.common.metric
org.apache.nemo.runtime.common.plan
org.apache.nemo.runtime.master

Uses of IRDAG in org.apache.nemo.client

Methods in org.apache.nemo.client with parameters of type IRDAG

Modifier and Type	Method	Description
static void	JobLauncher.launchDAG(IRDAG dag)	Launch application using the application DAG.
static void	JobLauncher.launchDAG(IRDAG dag, java.lang.String jobId)
static void	JobLauncher.launchDAG(IRDAG dag, java.util.Map<java.io.Serializable,java.lang.Object> broadcastVariables, java.lang.String jobId)

Uses of IRDAG in org.apache.nemo.common.ir

Method parameters in org.apache.nemo.common.ir with type arguments of type IRDAG

Modifier and Type	Method	Description
boolean	IRDAG.advanceDAGSnapshot(java.util.function.BiFunction<IRDAG,IRDAG,java.lang.Boolean> checker)	Used internally by Nemo to advance the DAG snapshot after applying each pass.
boolean	IRDAG.advanceDAGSnapshot(java.util.function.BiFunction<IRDAG,IRDAG,java.lang.Boolean> checker)	Used internally by Nemo to advance the DAG snapshot after applying each pass.

Uses of IRDAG in org.apache.nemo.common.pass

Methods in org.apache.nemo.common.pass that return types with arguments of type IRDAG

Modifier and Type	Method	Description
java.util.function.Predicate<IRDAG>	Pass.getCondition()	Getter for the condition under which to apply the pass.

Method parameters in org.apache.nemo.common.pass with type arguments of type IRDAG

Modifier and Type	Method	Description
Pass	Pass.addCondition(java.util.function.Predicate<IRDAG> newCondition)	Add the condition to the existing condition to run the pass.

Uses of IRDAG in org.apache.nemo.common.test

Methods in org.apache.nemo.common.test that return IRDAG

Modifier and Type	Method	Description
static IRDAG	EmptyComponents.buildEmptyDAG()	Builds dummy IR DAG for testing.
static IRDAG	EmptyComponents.buildEmptyDAGForSkew()	Builds dummy IR DAG to test skew handling.

Uses of IRDAG in org.apache.nemo.compiler

Methods in org.apache.nemo.compiler that return IRDAG

Modifier and Type	Method	Description
static IRDAG	CompilerTestUtil.compileALSDAG()
static IRDAG	CompilerTestUtil.compileALSInefficientDAG()
static IRDAG	CompilerTestUtil.compileMLRDAG()
static IRDAG	CompilerTestUtil.compileWordCountDAG()

Uses of IRDAG in org.apache.nemo.compiler.backend

Methods in org.apache.nemo.compiler.backend with parameters of type IRDAG

Modifier and Type	Method	Description
Plan	Backend.compile(IRDAG dag)	Compiles a DAG to a physical execution plan.

Uses of IRDAG in org.apache.nemo.compiler.backend.nemo

Methods in org.apache.nemo.compiler.backend.nemo with parameters of type IRDAG

Modifier and Type	Method	Description
PhysicalPlan	NemoBackend.compile(IRDAG irDAG)	Compiles an IR DAG into a PhysicalPlan to be submitted to Runtime.
void	NemoPlanRewriter.setCurrentIRDAG(IRDAG currentIRDAG)

Uses of IRDAG in org.apache.nemo.compiler.backend.nemo.prophet

Constructors in org.apache.nemo.compiler.backend.nemo.prophet with parameters of type IRDAG

Constructor	Description
ParallelismProphet(IRDAG irdag, PhysicalPlan physicalPlan, SimulationScheduler simulationScheduler, PhysicalPlanGenerator physicalPlanGenerator, java.util.Set<StageEdge> edgesToOptimize)	Default constructor for ParallelismProphet.

Uses of IRDAG in org.apache.nemo.compiler.frontend.beam

Methods in org.apache.nemo.compiler.frontend.beam that return IRDAG

Modifier and Type	Method	Description
IRDAG	PipelineVisitor.getConvertedPipeline()

Uses of IRDAG in org.apache.nemo.compiler.optimizer

Methods in org.apache.nemo.compiler.optimizer that return IRDAG

Modifier and Type	Method	Description
IRDAG	NemoOptimizer.optimizeAtCompileTime(IRDAG dag)
IRDAG	Optimizer.optimizeAtCompileTime(IRDAG dag)	Optimize the submitted DAG at compile time.
IRDAG	NemoOptimizer.optimizeAtRunTime(IRDAG dag, Message message)
IRDAG	Optimizer.optimizeAtRunTime(IRDAG dag, Message message)	Optimize the submitted DAG at run time.

Methods in org.apache.nemo.compiler.optimizer with parameters of type IRDAG

Modifier and Type	Method	Description
IRDAG	NemoOptimizer.optimizeAtCompileTime(IRDAG dag)
IRDAG	Optimizer.optimizeAtCompileTime(IRDAG dag)	Optimize the submitted DAG at compile time.
IRDAG	NemoOptimizer.optimizeAtRunTime(IRDAG dag, Message message)
IRDAG	Optimizer.optimizeAtRunTime(IRDAG dag, Message message)	Optimize the submitted DAG at run time.

Uses of IRDAG in org.apache.nemo.compiler.optimizer.pass.compiletime.annotating

Methods in org.apache.nemo.compiler.optimizer.pass.compiletime.annotating that return IRDAG

Modifier and Type	Method	Description
IRDAG	AggressiveSpeculativeCloningPass.apply(IRDAG dag)
IRDAG	CompressionPass.apply(IRDAG dag)
IRDAG	DefaultDataPersistencePass.apply(IRDAG dag)
IRDAG	DefaultDataStorePass.apply(IRDAG dag)
IRDAG	DefaultEdgeDecoderPass.apply(IRDAG dag)
IRDAG	DefaultEdgeEncoderPass.apply(IRDAG dag)
IRDAG	DefaultParallelismPass.apply(IRDAG dag)
IRDAG	DefaultScheduleGroupPass.apply(IRDAG dag)
IRDAG	DisaggregationEdgeDataStorePass.apply(IRDAG dag)
IRDAG	DuplicateEdgeGroupSizePass.apply(IRDAG dag)
IRDAG	LambdaPass.apply(IRDAG dag)
IRDAG	LargeShuffleAnnotatingPass.apply(IRDAG dag)
IRDAG	PipeTransferForAllEdgesPass.apply(IRDAG dag)
IRDAG	ResourceLocalityPass.apply(IRDAG dag)
IRDAG	ResourceSitePass.apply(IRDAG dag)
IRDAG	ResourceSlotPass.apply(IRDAG dag)
IRDAG	ShuffleEdgePushPass.apply(IRDAG dag)
IRDAG	SkewAnnotatingPass.apply(IRDAG dag)
IRDAG	TransientResourceDataTransferPass.apply(IRDAG dag)
IRDAG	TransientResourcePriorityPass.apply(IRDAG dag)
IRDAG	UpfrontCloningPass.apply(IRDAG dag)
IRDAG	XGBoostPass.apply(IRDAG dag)

Methods in org.apache.nemo.compiler.optimizer.pass.compiletime.annotating with parameters of type IRDAG

Modifier and Type	Method	Description
IRDAG	AggressiveSpeculativeCloningPass.apply(IRDAG dag)
IRDAG	CompressionPass.apply(IRDAG dag)
IRDAG	DefaultDataPersistencePass.apply(IRDAG dag)
IRDAG	DefaultDataStorePass.apply(IRDAG dag)
IRDAG	DefaultEdgeDecoderPass.apply(IRDAG dag)
IRDAG	DefaultEdgeEncoderPass.apply(IRDAG dag)
IRDAG	DefaultParallelismPass.apply(IRDAG dag)
IRDAG	DefaultScheduleGroupPass.apply(IRDAG dag)
IRDAG	DisaggregationEdgeDataStorePass.apply(IRDAG dag)
IRDAG	DuplicateEdgeGroupSizePass.apply(IRDAG dag)
IRDAG	LambdaPass.apply(IRDAG dag)
IRDAG	LargeShuffleAnnotatingPass.apply(IRDAG dag)
IRDAG	PipeTransferForAllEdgesPass.apply(IRDAG dag)
IRDAG	ResourceLocalityPass.apply(IRDAG dag)
IRDAG	ResourceSitePass.apply(IRDAG dag)
IRDAG	ResourceSlotPass.apply(IRDAG dag)
IRDAG	ShuffleEdgePushPass.apply(IRDAG dag)
IRDAG	SkewAnnotatingPass.apply(IRDAG dag)
IRDAG	TransientResourceDataTransferPass.apply(IRDAG dag)
IRDAG	TransientResourcePriorityPass.apply(IRDAG dag)
IRDAG	UpfrontCloningPass.apply(IRDAG dag)
IRDAG	XGBoostPass.apply(IRDAG dag)

Uses of IRDAG in org.apache.nemo.compiler.optimizer.pass.compiletime.composite

Methods in org.apache.nemo.compiler.optimizer.pass.compiletime.composite that return IRDAG

Modifier and Type	Method	Description
IRDAG	CompositePass.apply(IRDAG irVertexIREdgeDAG)

Methods in org.apache.nemo.compiler.optimizer.pass.compiletime.composite with parameters of type IRDAG

Modifier and Type	Method	Description
IRDAG	CompositePass.apply(IRDAG irVertexIREdgeDAG)

Uses of IRDAG in org.apache.nemo.compiler.optimizer.pass.compiletime.reshaping

Methods in org.apache.nemo.compiler.optimizer.pass.compiletime.reshaping that return IRDAG

Modifier and Type	Method	Description
IRDAG	CommonSubexpressionEliminationPass.apply(IRDAG inputDAG)
IRDAG	LargeShuffleReshapingPass.apply(IRDAG dag)
IRDAG	LoopExtractionPass.apply(IRDAG inputDAG)
IRDAG	LoopOptimizations.LoopFusionPass.apply(IRDAG inputDAG)
IRDAG	LoopOptimizations.LoopInvariantCodeMotionPass.apply(IRDAG inputDAG)
IRDAG	LoopUnrollingPass.apply(IRDAG inputDAG)
IRDAG	SamplingSkewReshapingPass.apply(IRDAG dag)
IRDAG	SamplingTaskSizingPass.apply(IRDAG dag)
IRDAG	SkewReshapingPass.apply(IRDAG dag)

Methods in org.apache.nemo.compiler.optimizer.pass.compiletime.reshaping with parameters of type IRDAG

Modifier and Type	Method	Description
IRDAG	CommonSubexpressionEliminationPass.apply(IRDAG inputDAG)
IRDAG	LargeShuffleReshapingPass.apply(IRDAG dag)
IRDAG	LoopExtractionPass.apply(IRDAG inputDAG)
IRDAG	LoopOptimizations.LoopFusionPass.apply(IRDAG inputDAG)
IRDAG	LoopOptimizations.LoopInvariantCodeMotionPass.apply(IRDAG inputDAG)
IRDAG	LoopUnrollingPass.apply(IRDAG inputDAG)
IRDAG	SamplingSkewReshapingPass.apply(IRDAG dag)
IRDAG	SamplingTaskSizingPass.apply(IRDAG dag)
IRDAG	SkewReshapingPass.apply(IRDAG dag)

Uses of IRDAG in org.apache.nemo.compiler.optimizer.pass.runtime

Methods in org.apache.nemo.compiler.optimizer.pass.runtime that return IRDAG

Modifier and Type	Method	Description
IRDAG	DynamicTaskSizingRuntimePass.apply(IRDAG irdag, Message<java.util.Map<java.lang.String,java.lang.Long>> mapMessage)
IRDAG	SkewRunTimePass.apply(IRDAG irdag, Message<java.util.Map<java.lang.Object,java.lang.Long>> message)

Methods in org.apache.nemo.compiler.optimizer.pass.runtime with parameters of type IRDAG

Modifier and Type	Method	Description
IRDAG	DynamicTaskSizingRuntimePass.apply(IRDAG irdag, Message<java.util.Map<java.lang.String,java.lang.Long>> mapMessage)
IRDAG	SkewRunTimePass.apply(IRDAG irdag, Message<java.util.Map<java.lang.Object,java.lang.Long>> message)

Uses of IRDAG in org.apache.nemo.compiler.optimizer.policy

Methods in org.apache.nemo.compiler.optimizer.policy that return IRDAG

Modifier and Type	Method	Description
IRDAG	BasicPullPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	BasicPushPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	ConditionalLargeShufflePolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DataSkewPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DefaultPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DefaultPolicyWithSeparatePass.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DisaggregationPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DynamicTaskSizingPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	LambdaPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	LargeShufflePolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	Policy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)	Optimize the DAG with the compile-time optimizations.
IRDAG	PolicyImpl.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	SamplingLargeShuffleSkewPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	StreamingPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	TestPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	TransientResourcePolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	XGBoostPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	BasicPullPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	BasicPushPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	ConditionalLargeShufflePolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DataSkewPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DefaultPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DefaultPolicyWithSeparatePass.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DisaggregationPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DynamicTaskSizingPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	LambdaPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	LargeShufflePolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	Policy.runRunTimeOptimizations(IRDAG dag, Message<?> message)	Optimize the DAG with the run-time optimizations.
IRDAG	PolicyImpl.runRunTimeOptimizations(IRDAG irdag, Message message)
IRDAG	SamplingLargeShuffleSkewPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	StreamingPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	TestPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	TransientResourcePolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	XGBoostPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)

Methods in org.apache.nemo.compiler.optimizer.policy with parameters of type IRDAG

Modifier and Type	Method	Description
IRDAG	BasicPullPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	BasicPushPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	ConditionalLargeShufflePolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DataSkewPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DefaultPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DefaultPolicyWithSeparatePass.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DisaggregationPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	DynamicTaskSizingPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	LambdaPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	LargeShufflePolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	Policy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)	Optimize the DAG with the compile-time optimizations.
IRDAG	PolicyImpl.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	SamplingLargeShuffleSkewPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	StreamingPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	TestPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	TransientResourcePolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	XGBoostPolicy.runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)
IRDAG	BasicPullPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	BasicPushPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	ConditionalLargeShufflePolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DataSkewPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DefaultPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DefaultPolicyWithSeparatePass.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DisaggregationPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	DynamicTaskSizingPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	LambdaPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	LargeShufflePolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	Policy.runRunTimeOptimizations(IRDAG dag, Message<?> message)	Optimize the DAG with the run-time optimizations.
IRDAG	PolicyImpl.runRunTimeOptimizations(IRDAG irdag, Message message)
IRDAG	SamplingLargeShuffleSkewPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	StreamingPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	TestPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	TransientResourcePolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)
IRDAG	XGBoostPolicy.runRunTimeOptimizations(IRDAG dag, Message<?> message)

Method parameters in org.apache.nemo.compiler.optimizer.policy with type arguments of type IRDAG

Modifier and Type	Method	Description
PolicyBuilder	PolicyBuilder.registerCompileTimePass(CompileTimePass compileTimePass, java.util.function.Predicate<IRDAG> condition)	Register compile time pass with its condition under which to run the pass.
PolicyBuilder	PolicyBuilder.registerRunTimePass(RunTimePass<?> runTimePass, CompileTimePass runTimePassRegisterer, java.util.function.Predicate<IRDAG> condition)	Register a run time pass.

Uses of IRDAG in org.apache.nemo.runtime.common.metric

Methods in org.apache.nemo.runtime.common.metric with parameters of type IRDAG

Modifier and Type	Method	Description
void	JobMetric.setIRDAG(IRDAG irDag)	Setter for the IR DAG.
static Pair<java.lang.String,java.lang.String>	MetricUtils.stringifyIRDAGProperties(IRDAG irdag)	Stringify execution properties of an IR DAG.

Uses of IRDAG in org.apache.nemo.runtime.common.plan

Methods in org.apache.nemo.runtime.common.plan with parameters of type IRDAG

Modifier and Type	Method	Description
DAG<Stage,StageEdge>	PhysicalPlanGenerator.apply(IRDAG irDAG)	Generates the PhysicalPlan to be executed.
java.util.Map<IRVertex,java.lang.Integer>	StagePartitioner.apply(IRDAG irDAG)
DAG<Stage,StageEdge>	PhysicalPlanGenerator.stagePartitionIrDAG(IRDAG irDAG)	We take the stage-partitioned DAG and create actual stage and stage edge objects to create a DAG of stages.

Uses of IRDAG in org.apache.nemo.runtime.master

Methods in org.apache.nemo.runtime.master with parameters of type IRDAG

Modifier and Type	Method	Description
void	RuntimeMaster.recordIRDAGMetrics(IRDAG irdag, java.lang.String planId)	Record IR DAG related metrics.