Package org.apache.nemo.runtime.executor.data.block

Class FileBlock<K extends java.io.Serializable>

java.lang.Object

org.apache.nemo.runtime.executor.data.block.FileBlock<K>

Type Parameters:

K - the key type of its partitions.

All Implemented Interfaces:

Block<K>

@NotThreadSafe
public final class FileBlock<K extends java.io.Serializable>
extends java.lang.Object
implements Block<K>

This class represents a block which is stored in (local or remote) file. Concurrent read is supported, but concurrent write is not supported.

Constructor Summary

Constructors

Constructor	Description
FileBlock(java.lang.String blockId, Serializer serializer, java.lang.String filePath, FileMetadata<K> metadata, MemoryPoolAssigner memoryPoolAssigner)	Constructor.

Method Summary

Modifier and Type	Method	Description
java.util.List<FileArea>	asFileAreas(KeyRange keyRange)	Retrieves the list of FileAreas for the specified KeyRange.
java.util.Optional<java.util.Map<K,java.lang.Long>>	commit()	Commits this block to prevent further write.
void	commitPartitions()	Commits all un-committed partitions.
void	deleteFile()	Deletes the file that contains this block data.
java.lang.String	getId()
boolean	isCommitted()
java.lang.Iterable<NonSerializedPartition<K>>	readPartitions(KeyRange keyRange)	Retrieves the partitions of this block from the file in a specific key range and deserializes it.
java.lang.Iterable<SerializedPartition<K>>	readSerializedPartitions(KeyRange keyRange)	Retrieves the SerializedPartitions in a specific key range.
void	write(K key, java.lang.Object element)	Writes an element to non-committed block.
void	writePartitions(java.lang.Iterable<NonSerializedPartition<K>> partitions)	Writes NonSerializedPartitions to this block.
void	writeSerializedPartitions(java.lang.Iterable<SerializedPartition<K>> partitions)	Writes SerializedPartitions to this block.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FileBlock

public FileBlock(java.lang.String blockId,
                 Serializer serializer,
                 java.lang.String filePath,
                 FileMetadata<K> metadata,
                 MemoryPoolAssigner memoryPoolAssigner)

Constructor.

Parameters:

blockId - the ID of this block.

serializer - the Serializer.

filePath - the path of the file that this block will be stored.

metadata - the metadata for this block.

memoryPoolAssigner - the MemoryPoolAssigner for memory allocation.

Method Detail

write

public void write(K key,
                  java.lang.Object element)

Writes an element to non-committed block. Invariant: This should not be invoked after this block is committed. Invariant: This method does not support concurrent write.

Specified by:

write in interface Block<K extends java.io.Serializable>

Parameters:

key - the key.

element - the element to write.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

writePartitions

public void writePartitions(java.lang.Iterable<NonSerializedPartition<K>> partitions)

Writes NonSerializedPartitions to this block. Invariant: This method does not support concurrent write.

Specified by:

writePartitions in interface Block<K extends java.io.Serializable>

Parameters:

partitions - the NonSerializedPartitions to write.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

writeSerializedPartitions

public void writeSerializedPartitions(java.lang.Iterable<SerializedPartition<K>> partitions)

Writes SerializedPartitions to this block. Invariant: This method does not support concurrent write.

Specified by:

writeSerializedPartitions in interface Block<K extends java.io.Serializable>

Parameters:

partitions - the SerializedPartitions to store.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

readPartitions

public java.lang.Iterable<NonSerializedPartition<K>> readPartitions(KeyRange keyRange)

Retrieves the partitions of this block from the file in a specific key range and deserializes it.

Specified by:

readPartitions in interface Block<K extends java.io.Serializable>

Parameters:

keyRange - the key range.

Returns:

an iterable of NonSerializedPartitions.

Throws:

BlockFetchException - for any error occurred while trying to fetch a block.

readSerializedPartitions

public java.lang.Iterable<SerializedPartition<K>> readSerializedPartitions(KeyRange keyRange)

Retrieves the SerializedPartitions in a specific key range. Invariant: This should not be invoked before this block is committed.

Specified by:

readSerializedPartitions in interface Block<K extends java.io.Serializable>

Parameters:

keyRange - the key range to retrieve.

Returns:

an iterable of SerializedPartitions.

Throws:

BlockFetchException - for any error occurred while trying to fetch a block.

asFileAreas

public java.util.List<FileArea> asFileAreas(KeyRange keyRange)
                                     throws java.io.IOException

Retrieves the list of FileAreas for the specified KeyRange.

Parameters:

keyRange - the key range

Returns:

list of the file areas

Throws:

java.io.IOException - if failed to open a file channel

deleteFile

public void deleteFile()
                throws java.io.IOException

Deletes the file that contains this block data. This method have to be called after all read is completed (or failed).

Throws:

java.io.IOException - if failed to delete.

commit

public java.util.Optional<java.util.Map<K,java.lang.Long>> commit()

Commits this block to prevent further write.

Specified by:

commit in interface Block<K extends java.io.Serializable>

Returns:

the size of each partition.

Throws:

BlockWriteException - for any error occurred while trying to write a block.

commitPartitions

public void commitPartitions()

Commits all un-committed partitions. The committed partitions will be flushed to the storage.

Specified by:

commitPartitions in interface Block<K extends java.io.Serializable>

getId

public java.lang.String getId()

Specified by:

getId in interface Block<K extends java.io.Serializable>

Returns:

the ID of this block.

isCommitted

public boolean isCommitted()

Specified by:

isCommitted in interface Block<K extends java.io.Serializable>

Returns:

whether this block is committed or not.