Package org.apache.nemo.common.ir.edge.executionproperty

Class PartitionSetProperty

java.lang.Object

org.apache.nemo.common.ir.executionproperty.ExecutionProperty<T>

org.apache.nemo.common.ir.executionproperty.EdgeExecutionProperty<java.util.ArrayList<KeyRange>>

org.apache.nemo.common.ir.edge.executionproperty.PartitionSetProperty

All Implemented Interfaces:

java.io.Serializable

public final class PartitionSetProperty
extends EdgeExecutionProperty<java.util.ArrayList<KeyRange>>

This property decides which partitions the tasks of the destination IRVertex should fetch. The position of a KeyRange in the list corresponds to the offset of the destination task.

For example, in the following setup: Source IRVertex (Parallelism=2) - IREdge (Partitioner.Num=4) - Destination IRVertex (Parallelism=2)

Setting PartitionSetProperty([0, 3), [3, 3)) on the IREdge with will enforce the following behaviors. - The first destination task fetches the first 3 partitions from each of the 2 data blocks - The second destination task fetches the last partitions from each of the 2 data blocks

This property is useful for handling data skews. For example, if the size ratio of the 4 partitions in the above setup are (17%, 16%, 17%, 50%), then each of the destination task will evenly handle 50% of the load.

See Also:

Serialized Form

Method Summary

Modifier and Type	Method	Description
static PartitionSetProperty	of(java.util.ArrayList<KeyRange> value)	Static method exposing the constructor.

Methods inherited from class org.apache.nemo.common.ir.executionproperty.ExecutionProperty

equals, getValue, hashCode, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Method Detail

of

public static PartitionSetProperty of(java.util.ArrayList<KeyRange> value)

Static method exposing the constructor.

Parameters:

value - value of the new execution property.

Returns:

the newly created execution property.