Package org.apache.nemo.compiler.frontend.beam.transform

Class PushBackDoFnTransform<InputT,OutputT>

java.lang.Object

org.apache.nemo.compiler.frontend.beam.transform.AbstractDoFnTransform<InputT,InputT,OutputT>

org.apache.nemo.compiler.frontend.beam.transform.PushBackDoFnTransform<InputT,OutputT>

Type Parameters:

InputT - input type.

OutputT - output type.

All Implemented Interfaces:

java.io.Serializable, Transform<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.util.WindowedValue<OutputT>>

public final class PushBackDoFnTransform<InputT,OutputT>
extends AbstractDoFnTransform<InputT,InputT,OutputT>

DoFn transform implementation with push backs for side inputs.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.nemo.common.ir.vertex.transform.Transform

Transform.Context

Constructor Summary

Constructors

Constructor

Description

PushBackDoFnTransform(org.apache.beam.sdk.transforms.DoFn<InputT,OutputT> doFn, org.apache.beam.sdk.coders.Coder<InputT> inputCoder, java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoders, org.apache.beam.sdk.values.TupleTag<OutputT> mainOutputTag, java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags, org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy, java.util.Map<java.lang.Integer,org.apache.beam.sdk.values.PCollectionView<?>> sideInputs, org.apache.beam.sdk.options.PipelineOptions options, org.apache.beam.sdk.transforms.display.DisplayData displayData, org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation, java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping)

PushBackDoFnTransform Constructor.

Method Summary

Modifier and Type	Method	Description
protected void	beforeClose()
void	onData(org.apache.beam.sdk.util.WindowedValue data)	On data received.
void	onWatermark(Watermark watermark)	On watermark received.
protected org.apache.beam.sdk.transforms.DoFn	wrapDoFn(org.apache.beam.sdk.transforms.DoFn initDoFn)

Methods inherited from class org.apache.nemo.compiler.frontend.beam.transform.AbstractDoFnTransform

close, getDoFn, getOutputCollector, onLatencymark, prepare, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

PushBackDoFnTransform

public PushBackDoFnTransform(org.apache.beam.sdk.transforms.DoFn<InputT,OutputT> doFn,
                             org.apache.beam.sdk.coders.Coder<InputT> inputCoder,
                             java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoders,
                             org.apache.beam.sdk.values.TupleTag<OutputT> mainOutputTag,
                             java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags,
                             org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy,
                             java.util.Map<java.lang.Integer,org.apache.beam.sdk.values.PCollectionView<?>> sideInputs,
                             org.apache.beam.sdk.options.PipelineOptions options,
                             org.apache.beam.sdk.transforms.display.DisplayData displayData,
                             org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation,
                             java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping)

PushBackDoFnTransform Constructor.

Parameters:

doFn - doFn

inputCoder - input coder

outputCoders - output coders

mainOutputTag - main output tag

additionalOutputTags - additional output tags

windowingStrategy - windowing strategy

sideInputs - side inputs

options - pipeline options

displayData - display data.

doFnSchemaInformation - DoFn schema information.

sideInputMapping - side input mapping.

Method Detail

wrapDoFn

protected org.apache.beam.sdk.transforms.DoFn wrapDoFn(org.apache.beam.sdk.transforms.DoFn initDoFn)

onData

public void onData(org.apache.beam.sdk.util.WindowedValue data)

Description copied from interface: Transform

On data received.

Parameters:

data - data received.

onWatermark

public void onWatermark(Watermark watermark)

Description copied from interface: Transform

On watermark received. This method should be called for the minimum watermark among input streams (input watermark). Transform may emit collected data after receiving watermarks.

Parameters:

watermark - watermark

beforeClose

protected void beforeClose()