Class StreamingScheduler

  • All Implemented Interfaces:
    Scheduler

    @NotThreadSafe
    public final class StreamingScheduler
    extends Object
    implements Scheduler
    A simple scheduler for streaming workloads. - Keeps track of new executors - Schedules all tasks in the plan at once. - Crashes the system upon any other events (should be fixed in the future) - Never stops running.
    • Method Detail

      • schedulePlan

        public void schedulePlan​(PhysicalPlan submittedPhysicalPlan,
                                 int maxScheduleAttempt)
        Description copied from interface: Scheduler
        Schedules the given plan.
        Specified by:
        schedulePlan in interface Scheduler
        Parameters:
        submittedPhysicalPlan - the plan of the job being submitted.
        maxScheduleAttempt - the max number of times this plan/sub-part of the plan should be attempted.
      • updatePlan

        public void updatePlan​(PhysicalPlan newPhysicalPlan)
        Description copied from interface: Scheduler
        Receives and updates the scheduler with a new physical plan for a job.
        Specified by:
        updatePlan in interface Scheduler
        Parameters:
        newPhysicalPlan - new physical plan for the job.
      • onTaskStateReportFromExecutor

        public void onTaskStateReportFromExecutor​(String executorId,
                                                  String taskId,
                                                  int taskAttemptIndex,
                                                  TaskState.State newState,
                                                  @Nullable
                                                  String vertexPutOnHold,
                                                  TaskState.RecoverableTaskFailureCause failureCause)
        Description copied from interface: Scheduler
        Called when a Task's execution state changes.
        Specified by:
        onTaskStateReportFromExecutor in interface Scheduler
        Parameters:
        executorId - of the executor in which the Task is executing.
        taskId - of the Task whose state must be updated.
        taskAttemptIndex - the number of times this Task has executed. ************** the below parameters are only valid for failures *****************
        newState - for the Task.
        vertexPutOnHold - the ID of task that are put on hold. It is null otherwise.
        failureCause - for which the Task failed in the case of a recoverable failure.
      • onSpeculativeExecutionCheck

        public void onSpeculativeExecutionCheck()
        Description copied from interface: Scheduler
        Called to check for speculative execution.
        Specified by:
        onSpeculativeExecutionCheck in interface Scheduler
      • onExecutorAdded

        public void onExecutorAdded​(ExecutorRepresenter executorRepresenter)
        Description copied from interface: Scheduler
        Called when an executor is added to Runtime, so that the extra resource can be used to execute the job.
        Specified by:
        onExecutorAdded in interface Scheduler
        Parameters:
        executorRepresenter - a representation of the added executor.
      • onExecutorRemoved

        public void onExecutorRemoved​(String executorId)
        Description copied from interface: Scheduler
        Called when an executor is removed from Runtime, so that faults related to the removal can be handled.
        Specified by:
        onExecutorRemoved in interface Scheduler
        Parameters:
        executorId - of the executor that has been removed.
      • terminate

        public void terminate()
        Description copied from interface: Scheduler
        To be called when a job should be terminated. Any clean up code should be implemented in this method.
        Specified by:
        terminate in interface Scheduler