Class BlockManagerWorker


  • @ThreadSafe
    public final class BlockManagerWorker
    extends java.lang.Object
    Executor-side block manager.
    • Method Detail

      • createBlock

        public Block createBlock​(java.lang.String blockId,
                                 DataStoreProperty.Value blockStore)
        Creates a new block.
        Parameters:
        blockId - the ID of the block to create.
        blockStore - the store to place the block.
        Returns:
        the created block.
      • readBlock

        public java.util.concurrent.CompletableFuture<DataUtil.IteratorWithNumBytes> readBlock​(java.lang.String blockIdWildcard,
                                                                                               java.lang.String runtimeEdgeId,
                                                                                               ExecutionPropertyMap<EdgeExecutionProperty> edgeProperties,
                                                                                               KeyRange keyRange)
        Inquiries the location of the specific block and routes the request to the local block manager worker or to the lower data plane. This can be invoked multiple times per blockId (maybe due to failures).
        Parameters:
        blockIdWildcard - of the block.
        runtimeEdgeId - id of the runtime edge that corresponds to the block.
        edgeProperties - for the edge.
        keyRange - the key range descriptor
        Returns:
        the CompletableFuture of the block.
      • writeBlock

        public void writeBlock​(Block block,
                               DataStoreProperty.Value blockStore,
                               int expectedReadTotal,
                               DataPersistenceProperty.Value persistence)
        Writes a block to a store.
        Parameters:
        block - the block to write.
        blockStore - the store to save the block.
        expectedReadTotal - the expected number of read for this block.
        persistence - how to handle the used block.
      • removeBlock

        public void removeBlock​(java.lang.String blockId,
                                DataStoreProperty.Value blockStore)
        Remove the block from store.
        Parameters:
        blockId - the ID of the block to remove.
        blockStore - the store which contains the block.
      • onOutputContext

        public void onOutputContext​(ByteOutputContext outputContext)
                             throws com.google.protobuf.InvalidProtocolBufferException
        Respond to a block request by another executor.

        This method is executed by {org.apache.nemo.runtime.executor.data.blocktransfer.BlockTransport} thread. \ Never execute a blocking call in this method!

        Parameters:
        outputContext - ByteOutputContext
        Throws:
        com.google.protobuf.InvalidProtocolBufferException - from errors during parsing context descriptor
      • onInputContext

        public void onInputContext​(ByteInputContext inputContext)
        Respond to a block notification by another executor.

        This method is executed by {org.apache.nemo.runtime.executor.data.blocktransfer.BlockTransport} thread. Never execute a blocking call in this method!

        Parameters:
        inputContext - ByteInputContext