Class PlanStateManager


  • @ThreadSafe
    public final class PlanStateManager
    extends Object
    Maintains three levels of state machines (PlanState, StageState, and TaskState) of a physical plan. The main API this class provides is onTaskStateReportFromExecutor(), which directly changes a TaskState. PlanState and StageState are updated internally in the class, and can only be read from the outside.

    (CONCURRENCY) The public methods of this class are synchronized.

    • Method Detail

      • updatePlan

        public void updatePlan​(PhysicalPlan physicalPlanToUpdate,
                               int maxScheduleAttemptToSet)
        Update the physical plan and maximum attempt.
        Parameters:
        physicalPlanToUpdate - the physical plan to manage.
        maxScheduleAttemptToSet - the maximum number of times this plan/sub-part of the plan should be attempted.
      • getTaskAttemptsToSchedule

        public List<String> getTaskAttemptsToSchedule​(String stageId)
        Get task attempts that are "READY".
        Parameters:
        stageId - to run
        Returns:
        executable task attempts
      • getAllTaskAttemptsOfStage

        public Set<String> getAllTaskAttemptsOfStage​(String stageId)
        Parameters:
        stageId - to query.
        Returns:
        all task attempt ids of the stage.
      • getExecutingTaskToRunningTimeMs

        public Map<String,​Long> getExecutingTaskToRunningTimeMs​(String stageId)
        Parameters:
        stageId - to query.
        Returns:
        a map from an EXECUTING task to its running time so far.
      • getCompletedTaskTimeListMs

        public List<Long> getCompletedTaskTimeListMs​(String stageId)
        List of task times so far for this stage.
        Parameters:
        stageId - of the stage.
        Returns:
        a copy of the list, empty if none completed.
      • setNumOfClones

        public boolean setNumOfClones​(String stageId,
                                      int taskIndex,
                                      int numOfClones)
        Parameters:
        stageId - of the clone.
        taskIndex - of the clone.
        numOfClones - of the clone.
        Returns:
        true if the numOfClones has been modified, false otherwise
      • onTaskStateChanged

        public void onTaskStateChanged​(String taskId,
                                       TaskState.State newTaskState)
        Updates the state of a task. Task state changes can occur both in master and executor. State changes that occur in master are initiated in BatchScheduler. State changes that occur in executors are sent to master as a control message, and the call to this method is initiated in BatchScheduler when the message/event is received.
        Parameters:
        taskId - the ID of the task.
        newTaskState - the new state of the task.
      • waitUntilFinish

        public PlanState.State waitUntilFinish()
        Wait for this plan to be finished and return the final state.
        Returns:
        the final state of this plan.
      • waitUntilFinish

        public PlanState.State waitUntilFinish​(long timeout,
                                               TimeUnit unit)
        Wait for this plan to be finished and return the final state. It wait for at most the given time.
        Parameters:
        timeout - of waiting.
        unit - of the timeout.
        Returns:
        the final state of this plan.
      • getAllTaskAttemptIdsToItsState

        public Map<String,​TaskState.State> getAllTaskAttemptIdsToItsState()
        Returns:
        a map from task attempt id to its current state.
      • isPlanDone

        public boolean isPlanDone()
        Returns:
        whether the execution for the plan is done or not.
      • getPlanId

        public String getPlanId()
        Returns:
        the ID of the plan.
      • getPlanState

        public PlanState.State getPlanState()
        Returns:
        the state of the plan.
      • getStageState

        public StageState.State getStageState​(String stageId)
        Parameters:
        stageId - the stage ID to query.
        Returns:
        the state of the stage.
      • getTaskState

        public TaskState.State getTaskState​(String taskId)
        Parameters:
        taskId - the ID of the task to query.
        Returns:
        the state of the task.
      • getPhysicalPlan

        public PhysicalPlan getPhysicalPlan()
        Returns:
        the physical plan.
      • getMaxScheduleAttempt

        public int getMaxScheduleAttempt()
        Returns:
        the maximum number of task scheduling.
      • isInitialized

        public boolean isInitialized()
        Returns:
        whether any plan has been submitted and initialized.
      • storeJSON

        public void storeJSON​(String suffix)
        Stores JSON representation of plan state into a file.
        Parameters:
        suffix - suffix for file name