Package org.apache.nemo.runtime.master.scheduler

Class BatchScheduler

  • All Implemented Interfaces:
    Scheduler
    @NotThreadSafe
public final class BatchScheduler
extends java.lang.Object
implements Scheduler
    (CONCURRENCY) Only a single dedicated thread should use the public methods of this class. (i.e., runtimeMasterThread in RuntimeMaster)

    BatchScheduler receives a single PhysicalPlan to execute and schedules the Tasks. Note: When modifying this class, take a look at SimulationScheduler.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void onExecutorAdded​(ExecutorRepresenter executorRepresenter)
      Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.
      void onExecutorRemoved​(java.lang.String executorId)
      Called when an executor is removed from Runtime, so that faults related to the removal can be handled.
      void onRunTimePassMessage​(java.lang.String taskId, java.lang.Object data)
      Process the RuntimePassMessage.
      void onSpeculativeExecutionCheck()
      Called to check for speculative execution.
      void onTaskStateReportFromExecutor​(java.lang.String executorId, java.lang.String taskId, int taskAttemptIndex, TaskState.State newState, java.lang.String vertexPutOnHold, TaskState.RecoverableTaskFailureCause failureCause)
      Handles task state transition notifications sent from executors.
      void schedulePlan​(PhysicalPlan submittedPhysicalPlan, int maxScheduleAttempt)
      Schedules a given plan.
      void terminate()
      To be called when a job should be terminated.
      void updatePlan​(PhysicalPlan newPhysicalPlan)
      Receives and updates the scheduler with a new physical plan for a job.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • updatePlan

        public void updatePlan​(PhysicalPlan newPhysicalPlan)
        Description copied from interface: Scheduler
        Receives and updates the scheduler with a new physical plan for a job.
        Specified by:
        updatePlan in interface Scheduler
        Parameters:
        newPhysicalPlan - new physical plan for the job.

      • onRunTimePassMessage

        public void onRunTimePassMessage​(java.lang.String taskId,
                                 java.lang.Object data)
        Process the RuntimePassMessage.
        Parameters:
        taskId - that generated the message.
        data - of the message.

      • schedulePlan

        public void schedulePlan​(PhysicalPlan submittedPhysicalPlan,
                         int maxScheduleAttempt)
        Schedules a given plan. If multiple physical plans are submitted, they will be appended and handled as a single plan. TODO #182: Consider reshaping in run-time optimization. At now, we only consider plan appending.
        Specified by:
        schedulePlan in interface Scheduler
        Parameters:
        submittedPhysicalPlan - the physical plan to schedule.
        maxScheduleAttempt - the max number of times this plan/sub-part of the plan should be attempted.

      • onTaskStateReportFromExecutor

        public void onTaskStateReportFromExecutor​(java.lang.String executorId,
                                          java.lang.String taskId,
                                          int taskAttemptIndex,
                                          TaskState.State newState,
                                          @Nullable
                                          java.lang.String vertexPutOnHold,
                                          TaskState.RecoverableTaskFailureCause failureCause)
        Handles task state transition notifications sent from executors. Note that we can receive notifications for previous task attempts, due to the nature of asynchronous events. We ignore such late-arriving notifications, and only handle notifications for the current task attempt.
        Specified by:
        onTaskStateReportFromExecutor in interface Scheduler
        Parameters:
        executorId - the id of the executor where the message was sent from.
        taskId - whose state has changed
        taskAttemptIndex - of the task whose state has changed
        newState - the state to change to
        vertexPutOnHold - the ID of vertex that is put on hold. It is null otherwise.
        failureCause - for which the Task failed in the case of a recoverable failure.

      • onSpeculativeExecutionCheck

        public void onSpeculativeExecutionCheck()
        Description copied from interface: Scheduler
        Called to check for speculative execution.
        Specified by:
        onSpeculativeExecutionCheck in interface Scheduler

      • onExecutorAdded

        public void onExecutorAdded​(ExecutorRepresenter executorRepresenter)
        Description copied from interface: Scheduler
        Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.
        Specified by:
        onExecutorAdded in interface Scheduler
        Parameters:
        executorRepresenter - a representation of the added executor.

      • onExecutorRemoved

        public void onExecutorRemoved​(java.lang.String executorId)
        Description copied from interface: Scheduler
        Called when an executor is removed from Runtime, so that faults related to the removal can be handled.
        Specified by:
        onExecutorRemoved in interface Scheduler
        Parameters:
        executorId - of the executor that has been removed.

      • terminate

        public void terminate()
        Description copied from interface: Scheduler
        To be called when a job should be terminated. Any clean up code should be implemented in this method.
        Specified by:
        terminate in interface Scheduler