Package com.illumon.iris.db.tables.dataimport.logtailer

Class SimpleDataImportStreamProcessor

java.lang.Object

com.illumon.iris.db.tables.dataimport.logtailer.DataImportProcessorBaseImpl

com.illumon.iris.db.tables.dataimport.logtailer.DataImportStreamProcessor

com.illumon.iris.db.tables.dataimport.logtailer.SimpleDataImportStreamProcessor

All Implemented Interfaces:

DataImportProcessorBase, ProcessDataLocker

public class SimpleDataImportStreamProcessor
extends DataImportStreamProcessor
implements ProcessDataLocker

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	SimpleDataImportStreamProcessor.OnReject	An object defined by the caller, capable of fulfilling reject(String, Exception)

Nested classes/interfaces inherited from class com.illumon.iris.db.tables.dataimport.logtailer.DataImportStreamProcessor

DataImportStreamProcessor.Context

Field Summary

Fields inherited from class com.illumon.iris.db.tables.dataimport.logtailer.DataImportProcessorBaseImpl

CHECKPOINT_INTERVAL_MILLIS_SUFFIX, dataStreamProcessorDirectory, DISP_DEFAULT_PREFIX, DISP_PREFIX, FLUSH_INTERVAL_SUFFIX, log, logPrefix, MAX_SHUTDOWN_DELAY_MILLIS_SUFFIX, PROFILING_LEVEL_SUFFIX, schemaService

Method Summary

Modifier and Type	Method	Description
protected Boolean	checkLiveness()	Test "liveness" for this stream processor.
void	close(String reason)
protected String	getDescription()
static SimpleDataImportStreamProcessor	make(com.fishlib.io.sched.Scheduler scheduler, com.fishlib.configuration.Configuration configuration, com.fishlib.io.logger.Logger log, String description, FullTableLocationKey streamKey, com.illumon.iris.db.tables.dataimport.logtailer.DataImportStreamProcessor.Directory streamProcessorDirectory, LocalTableDataService tableDataService, StringCacheProvider stringCacheProvider, SchemaService schemaService, Runnable closeCallback, SimpleDataImportStreamProcessor.OnReject onReject)	Factory method to create a new SimpleDataImportStreamProcessor
protected boolean	onInitialization()
void	processData(Runnable runnable)	Call this with your runnable that processes data and writes to the table to ensure proper locking around the checkpointing and flushing.
protected void	reject(String reason, Exception cause)
void	runWhenConsistent(Runnable runnable)	Wait for the update counter to be consistent, and then run the given function.
SafeCloseable	suspendCheckpointing()	Prevent checkpoints from being written until the returned Closeable is closed.

Methods inherited from class com.illumon.iris.db.tables.dataimport.logtailer.DataImportStreamProcessor

addPostCommitCallback, addPostPrepareCallback, initialize, shutdown, toString

Methods inherited from class com.illumon.iris.db.tables.dataimport.logtailer.DataImportProcessorBaseImpl

getLog, getLogPrefix

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

make

public static SimpleDataImportStreamProcessor make(@NotNull
 com.fishlib.io.sched.Scheduler scheduler,
 @NotNull
 com.fishlib.configuration.Configuration configuration,
 @NotNull
 com.fishlib.io.logger.Logger log,
 @NotNull
 String description,
 @NotNull
 FullTableLocationKey streamKey,
 @NotNull
 com.illumon.iris.db.tables.dataimport.logtailer.DataImportStreamProcessor.Directory streamProcessorDirectory,
 @NotNull
 LocalTableDataService tableDataService,
 @NotNull
 StringCacheProvider stringCacheProvider,
 @NotNull
 SchemaService schemaService,
 @NotNull
 Runnable closeCallback,
 @NotNull
 SimpleDataImportStreamProcessor.OnReject onReject)

Factory method to create a new SimpleDataImportStreamProcessor

Parameters:

scheduler - The scheduler for the flush and commit tasks

configuration - the configuration to use

log - The log

description - description of the data connection

streamKey - Destination information

streamProcessorDirectory - An object to lookup Processors based on table key.

tableDataService - The table data service

stringCacheProvider - The string cache provider

schemaService - The schemaService

closeCallback - callback to be invoked when the simple data stream is closed

onReject - callback to be invoked if the simple data stream is rejected

Returns:

a new SimpleDataImportStreamProcessor

processData

public void processData(@NotNull
 Runnable runnable)

Call this with your runnable that processes data and writes to the table to ensure proper locking around the checkpointing and flushing.

Specified by:

processData in interface ProcessDataLocker

Parameters:

runnable - a runnable that processes data for this stream

onInitialization

protected boolean onInitialization()

close

public void close(@NotNull
 String reason)

checkLiveness

protected Boolean checkLiveness()

Description copied from class: DataImportStreamProcessor

Test "liveness" for this stream processor. checkLiveness will return one of three values:

• true - the current connection is known to be live - the heartbeat succeeded
• false - the current connection is dead - the heartbeat attempt threw an exception and the current channel has already been closed, or the liveness check timed out and the connection was killed
• null - the connection state is unknown - e.g. the heartbeat timed out the connection could not be killed

Specified by:

checkLiveness in class DataImportStreamProcessor

Returns:

True if alive, false if not, null if uncertain

getDescription

@NotNull
protected String getDescription()

reject

protected void reject(@NotNull
 String reason,
 @NotNull
 Exception cause)

suspendCheckpointing

public SafeCloseable suspendCheckpointing()

Prevent checkpoints from being written until the returned Closeable is closed.

Note: If you do not close the returned value; then data is never made durable.

Returns:

a Closeable object that resumes checkpoint processing.

runWhenConsistent

public void runWhenConsistent(Runnable runnable)

Wait for the update counter to be consistent, and then run the given function.

Parameters:

runnable - the runnable to execute under the processing lock; while the state is consistent