Class SamplingLargeShuffleSkewPolicy

  • All Implemented Interfaces:
    Serializable, Policy

    public final class SamplingLargeShuffleSkewPolicy
    extends Object
    implements Policy
    A policy to demonstrate the large shuffle optimization, witch batches disk seek during data shuffle.
    See Also:
    Serialized Form
    • Constructor Detail

      • SamplingLargeShuffleSkewPolicy

        public SamplingLargeShuffleSkewPolicy()
        Default constructor.
    • Method Detail

      • runCompileTimeOptimization

        public IRDAG runCompileTimeOptimization​(IRDAG dag,
                                                String dagDirectory)
        Description copied from interface: Policy
        Optimize the DAG with the compile-time optimizations.
        Specified by:
        runCompileTimeOptimization in interface Policy
        Parameters:
        dag - input DAG.
        dagDirectory - directory to save the DAG information.
        Returns:
        optimized DAG, reshaped or tagged with execution properties.
      • runRunTimeOptimizations

        public IRDAG runRunTimeOptimizations​(IRDAG dag,
                                             Message<?> message)
        Description copied from interface: Policy
        Optimize the DAG with the run-time optimizations.
        Specified by:
        runRunTimeOptimizations in interface Policy
        Parameters:
        dag - input DAG.
        message - from the DAG execution.
        Returns:
        optimized DAG, reshaped or tagged with execution properties.