Package org.apache.nemo.runtime.master

Class RuntimeMaster

  • public final class RuntimeMaster
extends java.lang.Object
    (WARNING) Use runtimeMasterThread for all public methods to avoid race conditions. See comments in the Scheduler for avoiding race conditions.

    Runtime Master is the central controller of Runtime. Compiler submits an PhysicalPlan to Runtime Master to execute a job. Runtime Master handles: a) Scheduling the plan with Scheduler. b) Managing resources with ContainerManager. c) Managing blocks with BlockManagerMaster. d) Receiving and sending control messages with MessageEnvironment. e) Metric using MetricMessageHandler.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      Pair<PlanStateManager,​java.util.concurrent.ScheduledExecutorService> execute​(PhysicalPlan plan, int maxScheduleAttempt)
      Submits the PhysicalPlan to Runtime.
      void flushMetrics()
      Flush metrics.
      void onContainerAllocated​(java.lang.String executorId, org.apache.reef.driver.evaluator.AllocatedEvaluator allocatedEvaluator, org.apache.reef.tang.Configuration executorConfiguration)
      Called when a container is allocated for this runtime.
      void onExecutorFailed​(org.apache.reef.driver.evaluator.FailedEvaluator failedEvaluator)
      Called when an executor fails due to container failure on this runtime.
      boolean onExecutorLaunched​(org.apache.reef.driver.context.ActiveContext activeContext)
      Called when an executor is launched on a container for this runtime.
      void recordIRDAGMetrics​(IRDAG irdag, java.lang.String planId)
      Record IR DAG related metrics.
      void requestContainer​(java.lang.String resourceSpecificationString)
      Requests a container with resource specification.
      void terminate()
      Terminates the RuntimeMaster.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • recordIRDAGMetrics

        public void recordIRDAGMetrics​(IRDAG irdag,
                               java.lang.String planId)
        Record IR DAG related metrics.
        Parameters:
        irdag - the IR DAG to record.
        planId - the ID of the IR DAG Physical Plan.

      • flushMetrics

        public void flushMetrics()
        Flush metrics.

      • execute

        public Pair<PlanStateManager,​java.util.concurrent.ScheduledExecutorService> execute​(PhysicalPlan plan,
                                                                                          int maxScheduleAttempt)
        Submits the PhysicalPlan to Runtime. At now, we are assuming that a single job submit multiple plans.
        Parameters:
        plan - to execute
        maxScheduleAttempt - the max number of times this plan/sub-part of the plan should be attempted.
        Returns:
        pair of PlanStateManager and ScheduledExecutorService

      • terminate

        public void terminate()
        Terminates the RuntimeMaster.

      • requestContainer

        public void requestContainer​(java.lang.String resourceSpecificationString)
        Requests a container with resource specification.
        Parameters:
        resourceSpecificationString - the resource specification.

      • onContainerAllocated

        public void onContainerAllocated​(java.lang.String executorId,
                                 org.apache.reef.driver.evaluator.AllocatedEvaluator allocatedEvaluator,
                                 org.apache.reef.tang.Configuration executorConfiguration)
        Called when a container is allocated for this runtime. A wrapper function for ContainerManager.
        Parameters:
        executorId - to use for the executor to be launched on this container.
        allocatedEvaluator - to be used as the container.
        executorConfiguration - to use for the executor to be launched on this container.

      • onExecutorLaunched

        public boolean onExecutorLaunched​(org.apache.reef.driver.context.ActiveContext activeContext)
        Called when an executor is launched on a container for this runtime.
        Parameters:
        activeContext - of the launched executor.
        Returns:
        true if all requested executors have been launched, false otherwise.

      • onExecutorFailed

        public void onExecutorFailed​(org.apache.reef.driver.evaluator.FailedEvaluator failedEvaluator)
        Called when an executor fails due to container failure on this runtime.
        Parameters:
        failedEvaluator - that failed.