com.mulesoft.mule.runtime.module.batch.internal.engine.buffer

Class StreamingAggregatorBuffer

java.lang.Object

com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.AbstractRecordBuffer

com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.InMemoryRecordBuffer

com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.AggregatorRecordBuffer

com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.StreamingAggregatorBuffer

All Implemented Interfaces:

RecordBuffer, org.mule.runtime.api.meta.NamedObject

public class StreamingAggregatorBuffer
extends AggregatorRecordBuffer

An implementation of AggregatorRecordBuffer for streaming instances of DefaultBatchStepAggregator. This implementation accumulates all the records in the current BatchJobInstanceAdapter into an input queue. Once the input queue has been filled with all the records, then this buffer flushes them using a streaming Iterator. As this iterator is consumed, processed records are accumulated into an output queue. Once iteration is completed, the records are finally sent to the next BatchStep. The reason for the output queue instead of flushing directly to the next step is functional: a) we don't want any record to be available to the next step before processing finishes and b) if one record fails we want to fail them all

Since:

3.5.0

Field Summary

Fields inherited from class com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.AggregatorRecordBuffer

batchEngine, exceptionReporter, muleContext, notificationDispatcher, step

Constructor Summary

Constructors

Constructor and Description
StreamingAggregatorBuffer(BatchEngine batchEngine, BatchStepAdapter step, org.mule.runtime.core.api.processor.Processor chain, org.mule.runtime.api.notification.NotificationDispatcher notificationDispatcher, org.mule.runtime.core.api.MuleContext muleContext)

Method Summary

Modifier and Type	Method and Description
int	add(BatchJobInstanceAdapter jobInstance, BatchTransactionContext ctx, Record record) buffers the given Record for the given BatchJobInstanceAdapter
protected org.mule.runtime.core.api.event.CoreEvent	configureChainEvent(org.mule.runtime.core.api.event.CoreEvent event, BatchJobInstanceAdapter jobInstance, com.google.common.collect.Multimap<BatchTransactionContext,Record> records) Configures the event (set payload, properties, etc) to is going to be used when flushing the aggregated records through the AggregatorRecordBuffer.chain
void	flush(BatchJobInstanceAdapter jobInstance) Flushes the buffered Records for the given BatchJobInstanceAdapter.
void	flushAndForget(BatchJobInstanceAdapter jobInstance) Atomically performs a InMemoryRecordBuffer.flush(com.mulesoft.mule.runtime.module.batch.engine.BatchJobInstanceAdapter) followed by a InMemoryRecordBuffer.forget(com.mulesoft.mule.runtime.module.batch.engine.BatchJobInstanceAdapter).
void	forget(BatchJobInstanceAdapter jobInstance) Releases all tracking information for the given BatchJobInstanceAdapter
protected BatchProcessingTemplate	makeProcessingTemplate(com.google.common.collect.Multimap<BatchTransactionContext,Record> records, org.mule.runtime.core.api.processor.Processor processor, org.mule.runtime.core.api.MuleContext muleContext) Returns the BatchProcessingTemplate through which AggregatorRecordBuffer.chain will be executed
long	size(BatchJobInstanceAdapter jobInstance) Returns how many records are buffered for the given BatchJobInstanceAdapter

Methods inherited from class com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.AggregatorRecordBuffer

doFlush, getLocation, markError, setExceptionReporter

Methods inherited from class com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.InMemoryRecordBuffer

getHolderLock

Methods inherited from class com.mulesoft.mule.runtime.module.batch.internal.engine.buffer.AbstractRecordBuffer

add, getName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamingAggregatorBuffer

public StreamingAggregatorBuffer(BatchEngine batchEngine,
                                 BatchStepAdapter step,
                                 org.mule.runtime.core.api.processor.Processor chain,
                                 org.mule.runtime.api.notification.NotificationDispatcher notificationDispatcher,
                                 org.mule.runtime.core.api.MuleContext muleContext)

Method Detail

add

public int add(BatchJobInstanceAdapter jobInstance,
               BatchTransactionContext ctx,
               Record record)

Description copied from class: InMemoryRecordBuffer

buffers the given Record for the given BatchJobInstanceAdapter

Specified by:

add in interface RecordBuffer

Overrides:

add in class InMemoryRecordBuffer

Parameters:

jobInstance - a BatchJobInstanceAdapter

record - a BatchJobInstanceAdapter

Returns:

how many records are still buffered for the given job instance

forget

public void forget(BatchJobInstanceAdapter jobInstance)

Description copied from class: InMemoryRecordBuffer

Releases all tracking information for the given BatchJobInstanceAdapter

Specified by:

forget in interface RecordBuffer

Overrides:

forget in class InMemoryRecordBuffer

Parameters:

jobInstance - a BatchJobInstanceAdapter which you no longer need to track

flushAndForget

public void flushAndForget(BatchJobInstanceAdapter jobInstance)

Description copied from class: InMemoryRecordBuffer

Atomically performs a InMemoryRecordBuffer.flush(com.mulesoft.mule.runtime.module.batch.engine.BatchJobInstanceAdapter) followed by a InMemoryRecordBuffer.forget(com.mulesoft.mule.runtime.module.batch.engine.BatchJobInstanceAdapter).

You MUST use this method when you know that the given jobInstance will no longer be executed in order to avoid a memory leak on its tracking information

This is NOT a thread-safe operation. You must be careful to not have several threads invoking this method at the same time for the same BatchJobInstanceAdapter

Specified by:

flushAndForget in interface RecordBuffer

Overrides:

flushAndForget in class InMemoryRecordBuffer

Parameters:

jobInstance - a BatchJobInstanceAdapter which you no longer need to track

flush

public void flush(BatchJobInstanceAdapter jobInstance)

Description copied from class: InMemoryRecordBuffer

Flushes the buffered Records for the given BatchJobInstanceAdapter.

Use this method when you want to flush the info but you still expect more records from the same job instance to come. In this way, you flush the records but you avoid constantly recreating the internal tracking information.

This is a thread-safe operation.

Specified by:

flush in interface RecordBuffer

Overrides:

flush in class InMemoryRecordBuffer

Parameters:

jobInstance - the BatchJobInstanceAdapter which records you want flushed

size

public long size(BatchJobInstanceAdapter jobInstance)

Description copied from class: InMemoryRecordBuffer

Returns how many records are buffered for the given BatchJobInstanceAdapter

Specified by:

size in interface RecordBuffer

Overrides:

size in class InMemoryRecordBuffer

Parameters:

jobInstance - a BatchJobInstanceAdapter

Returns:

a long value

configureChainEvent

protected org.mule.runtime.core.api.event.CoreEvent configureChainEvent(org.mule.runtime.core.api.event.CoreEvent event,
                                                                        BatchJobInstanceAdapter jobInstance,
                                                                        com.google.common.collect.Multimap<BatchTransactionContext,Record> records)
                                                                 throws org.mule.runtime.api.exception.MuleException

Description copied from class: AggregatorRecordBuffer

Configures the event (set payload, properties, etc) to is going to be used when flushing the aggregated records through the AggregatorRecordBuffer.chain

Specified by:

configureChainEvent in class AggregatorRecordBuffer

Parameters:

event - a CoreEvent

jobInstance - the BatchJobInstanceAdapter being executed

records - the aggregated Records

Returns:

a configured CoreEvent or null, which means that the chain should be skipped for this event

Throws:

org.mule.runtime.api.exception.MuleException - if anything goes wrong

makeProcessingTemplate

protected BatchProcessingTemplate makeProcessingTemplate(com.google.common.collect.Multimap<BatchTransactionContext,Record> records,
                                                         org.mule.runtime.core.api.processor.Processor processor,
                                                         org.mule.runtime.core.api.MuleContext muleContext)

Description copied from class: AggregatorRecordBuffer

Returns the BatchProcessingTemplate through which AggregatorRecordBuffer.chain will be executed

Specified by:

makeProcessingTemplate in class AggregatorRecordBuffer

Parameters:

records - the aggregated Records

processor - the Processor to be executed

muleContext - the current MuleContext

Returns: