Package org.apache.nemo.runtime.master

Class BlockManagerMaster

java.lang.Object

org.apache.nemo.runtime.master.BlockManagerMaster

@ThreadSafe
public final class BlockManagerMaster
extends java.lang.Object

Master-side block manager. This implementation assumes that only a single user application can submit (maybe multiple) plans through Scheduler.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	BlockManagerMaster.BlockManagerMasterControlMessageReceiver	Handler for control messages received.
static class	BlockManagerMaster.BlockRequestHandler	The handler of block location requests.

Method Summary

Modifier and Type	Method	Description
static BlockState.State	convertBlockState(ControlMessage.BlockStateFromExecutor state)	Return the corresponding BlockState.State for the specified ControlMessage.BlockStateFromExecutor.
static ControlMessage.BlockStateFromExecutor	convertBlockState(BlockState.State state)	Return the corresponding ControlMessage.BlockStateFromExecutor for the specified BlockState.State.
java.util.List<BlockManagerMaster.BlockRequestHandler>	getBlockHandlers(java.lang.String blockIdOrWildcard, BlockState.State state)	Get handlers of blocks that are in a particular state.
java.util.Set<java.lang.String>	getProducerTaskIds(java.lang.String blockId)	Gets the ids of the tasks which already produced or will produce data for a specific block.
void	onBlockStateChanged(java.lang.String blockId, BlockState.State newState, java.lang.String location)	Deals with state change of a block.
void	onProducerTaskFailed(java.lang.String failedTaskId)	To be called when a potential producer task fails.
void	onProducerTaskScheduled(java.lang.String taskId, java.util.Set<java.lang.String> blockIds)	To be called when a potential producer task is scheduled.
java.util.Set<java.lang.String>	removeWorker(java.lang.String executorId)	Manages the block information when a executor is removed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

removeWorker

public java.util.Set<java.lang.String> removeWorker(java.lang.String executorId)

Manages the block information when a executor is removed.

Parameters:

executorId - the id of removed executor.

Returns:

the set of tasks have to be recomputed.

getBlockHandlers

public java.util.List<BlockManagerMaster.BlockRequestHandler> getBlockHandlers(java.lang.String blockIdOrWildcard,
                                                                               BlockState.State state)

Get handlers of blocks that are in a particular state.

Parameters:

blockIdOrWildcard - to query

state - of the block

Returns:

the handlers, empty if none matches.

getProducerTaskIds

public java.util.Set<java.lang.String> getProducerTaskIds(java.lang.String blockId)

Gets the ids of the tasks which already produced or will produce data for a specific block.

Parameters:

blockId - the id of the block.

Returns:

the ids of the producer tasks.

onProducerTaskScheduled

public void onProducerTaskScheduled(java.lang.String taskId,
                                    java.util.Set<java.lang.String> blockIds)

To be called when a potential producer task is scheduled.

Parameters:

taskId - the ID of the scheduled task.

blockIds - this task will produce

onProducerTaskFailed

public void onProducerTaskFailed(java.lang.String failedTaskId)

To be called when a potential producer task fails. Only the Tasks that have not yet completed (i.e. blocks not yet committed) will call this method.

Parameters:

failedTaskId - the ID of the task that failed.

onBlockStateChanged

public void onBlockStateChanged(java.lang.String blockId,
                                BlockState.State newState,
                                @Nullable
                                java.lang.String location)

Deals with state change of a block.

Parameters:

blockId - the id of the block.

newState - the new state of the block.

location - the location of the block (e.g., worker id, remote store). null if not committed or lost.

convertBlockState

public static BlockState.State convertBlockState(ControlMessage.BlockStateFromExecutor state)

Return the corresponding BlockState.State for the specified ControlMessage.BlockStateFromExecutor.

Parameters:

state - ControlMessage.BlockStateFromExecutor

Returns:

the corresponding BlockState.State

convertBlockState

public static ControlMessage.BlockStateFromExecutor convertBlockState(BlockState.State state)

Return the corresponding ControlMessage.BlockStateFromExecutor for the specified BlockState.State.

Parameters:

state - BlockState.State

Returns:

the corresponding ControlMessage.BlockStateFromExecutor