Package org.apache.nemo.compiler.optimizer.policy

Class SamplingLargeShuffleSkewPolicy

java.lang.Object

org.apache.nemo.compiler.optimizer.policy.SamplingLargeShuffleSkewPolicy

All Implemented Interfaces:

java.io.Serializable, Policy

public final class SamplingLargeShuffleSkewPolicy
extends java.lang.Object
implements Policy

A policy to demonstrate the large shuffle optimization, witch batches disk seek during data shuffle.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field	Description
static PolicyBuilder	BUILDER

Constructor Summary

Constructors

Constructor	Description
SamplingLargeShuffleSkewPolicy()	Default constructor.

Method Summary

Modifier and Type	Method	Description
IRDAG	runCompileTimeOptimization(IRDAG dag, java.lang.String dagDirectory)	Optimize the DAG with the compile-time optimizations.
IRDAG	runRunTimeOptimizations(IRDAG dag, Message<?> message)	Optimize the DAG with the run-time optimizations.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

BUILDER

public static final PolicyBuilder BUILDER

Constructor Detail

SamplingLargeShuffleSkewPolicy

public SamplingLargeShuffleSkewPolicy()

Default constructor.

Method Detail

runCompileTimeOptimization

public IRDAG runCompileTimeOptimization(IRDAG dag,
                                        java.lang.String dagDirectory)

Description copied from interface: Policy

Optimize the DAG with the compile-time optimizations.

Specified by:

runCompileTimeOptimization in interface Policy

Parameters:

dag - input DAG.

dagDirectory - directory to save the DAG information.

Returns:

optimized DAG, reshaped or tagged with execution properties.

runRunTimeOptimizations

public IRDAG runRunTimeOptimizations(IRDAG dag,
                                     Message<?> message)

Description copied from interface: Policy

Optimize the DAG with the run-time optimizations.

Specified by:

runRunTimeOptimizations in interface Policy

Parameters:

dag - input DAG.

message - from the DAG execution.

Returns:

optimized DAG, reshaped or tagged with execution properties.