Interface Scheduler

  • All Known Implementing Classes:
    BatchScheduler, StreamingScheduler

    public interface Scheduler
    Only two threads call scheduling code: RuntimeMaster thread (RMT), and SchedulerThread(ST). RMT and ST meet only at two points: ExecutorRegistry, and PendingTaskCollectionPointer, which are synchronized(ThreadSafe). Other scheduler-related classes that are accessed by only one of the two threads are not synchronized(NotThreadSafe).
    • Method Detail

      • schedulePlan

        void schedulePlan​(PhysicalPlan physicalPlan,
                          int maxScheduleAttempt)
        Schedules the given plan.
        Parameters:
        physicalPlan - the plan of the job being submitted.
        maxScheduleAttempt - the max number of times this plan/sub-part of the plan should be attempted.
      • updatePlan

        void updatePlan​(PhysicalPlan newPhysicalPlan)
        Receives and updates the scheduler with a new physical plan for a job.
        Parameters:
        newPhysicalPlan - new physical plan for the job.
      • onExecutorAdded

        void onExecutorAdded​(ExecutorRepresenter executorRepresenter)
        Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.
        Parameters:
        executorRepresenter - a representation of the added executor.
      • onExecutorRemoved

        void onExecutorRemoved​(String executorId)
        Called when an executor is removed from Runtime, so that faults related to the removal can be handled.
        Parameters:
        executorId - of the executor that has been removed.
      • onTaskStateReportFromExecutor

        void onTaskStateReportFromExecutor​(String executorId,
                                           String taskId,
                                           int attemptIdx,
                                           TaskState.State newState,
                                           @Nullable
                                           String taskPutOnHold,
                                           TaskState.RecoverableTaskFailureCause failureCause)
        Called when a Task's execution state changes.
        Parameters:
        executorId - of the executor in which the Task is executing.
        taskId - of the Task whose state must be updated.
        newState - for the Task.
        attemptIdx - the number of times this Task has executed. ************** the below parameters are only valid for failures *****************
        taskPutOnHold - the ID of task that are put on hold. It is null otherwise.
        failureCause - for which the Task failed in the case of a recoverable failure.
      • onSpeculativeExecutionCheck

        void onSpeculativeExecutionCheck()
        Called to check for speculative execution.
      • terminate

        void terminate()
        To be called when a job should be terminated. Any clean up code should be implemented in this method.