Package com.illumon.iris.db.tables.dataimport.logtailer

Class DataImportServer

java.lang.Object

com.fishlib.util.process.OnetimeShutdownTask

com.illumon.iris.db.tables.dataimport.logtailer.DataImportServer

All Implemented Interfaces:

com.fishlib.util.process.ShutdownManager.Task, IDataImportServer

public class DataImportServer
extends com.fishlib.util.process.OnetimeShutdownTask
implements IDataImportServer

Data Import Server: Consumes row-oriented data from log tailers (or similar data sources) and persists Iris' column-oriented table format, simultaneously serving said data consistently to remote table data clients.

Field Summary

Fields

Modifier and Type	Field	Description
static final long	LOG_TAILER_ACK_TIMEOUT_DEFAULT
static final String	LOG_TAILER_ACK_TIMEOUT_PROP
static final long	LOG_TAILER_POLL_PAUSE_DEFAULT
static final String	LOG_TAILER_POLL_PAUSE_PROP

Constructor Summary

Constructors

Constructor	Description
DataImportServer(com.fishlib.io.logger.Logger logNotUsed, DataImportServiceConfig disConfig, com.fishlib.configuration.Configuration configuration)	Deprecated. use DataImportServer(DataImportServerLogFactory, DataImportServiceConfig, Configuration, SchemaService)
DataImportServer(DataImportServerLogFactory logFactory, DataImportServiceConfig disConfig, com.fishlib.configuration.Configuration configuration, SchemaService schemaService)	Construct a DataImportServer.

Method Summary

Modifier and Type	Method	Description
SimpleDataImportStreamProcessor	createSimpleProcessor(String description, FullTableLocationKey streamKey, Runnable closeCallback, SimpleDataImportStreamProcessor.OnReject onReject)	Create a SimpleDataImportStreamProcessor for importing data into this DataImportServer.
static DataImportServer	getDataImportServer(com.fishlib.io.logger.Logger log, DataImportServiceConfig disConfig, com.fishlib.configuration.Configuration configuration, SchemaService schemaService)	Deprecated.
static DataImportServer	getDataImportServer(DataImportServerLogFactory logCreator, DataImportServiceConfig disConfig, com.fishlib.configuration.Configuration configuration, SchemaService schemaService)	Create a new DataImportServer instance, according to the configuration passed in.
SchemaService	getSchemaService()
TableDataService	getTableDataService()
static void	main(String... args)
void	shutdown()
void	start()	Deprecated. use startInWorker() instead
void	start(com.fishlib.stats.StatsIntradayLogger statsIntradayLogger)	Beginning tracking statistics and accepting connections.
void	startInWorker()	Start accepting connections in-worker.

Methods inherited from class com.fishlib.util.process.OnetimeShutdownTask

adapt, awaitShutdown, awaitShutdown, invoke, isShutdown

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.illumon.iris.db.tables.dataimport.logtailer.IDataImportServer

isShutdown

Field Details

LOG_TAILER_POLL_PAUSE_PROP

public static final String LOG_TAILER_POLL_PAUSE_PROP

See Also:

• Constant Field Values

LOG_TAILER_ACK_TIMEOUT_PROP

public static final String LOG_TAILER_ACK_TIMEOUT_PROP

See Also:

• Constant Field Values

LOG_TAILER_POLL_PAUSE_DEFAULT

public static final long LOG_TAILER_POLL_PAUSE_DEFAULT

See Also:

• Constant Field Values

LOG_TAILER_ACK_TIMEOUT_DEFAULT

public static final long LOG_TAILER_ACK_TIMEOUT_DEFAULT

See Also:

• Constant Field Values

Constructor Details

DataImportServer

public DataImportServer(@Nullable
 DataImportServerLogFactory logFactory,
 @NotNull
 DataImportServiceConfig disConfig,
 @NotNull
 com.fishlib.configuration.Configuration configuration,
 @NotNull
 SchemaService schemaService)
                 throws IOException

Construct a DataImportServer.

Parameters:

logFactory - optional IrisLogCreator for access to loggers

disConfig - the parameters that define this DIS instance

configuration - the configuration to use when fetching settings that aren't included in disConfig

schemaService - the SchemaService to use

Throws:

IOException

DataImportServer

@Deprecated
public DataImportServer(@Nullable
 com.fishlib.io.logger.Logger logNotUsed,
 @NotNull
 DataImportServiceConfig disConfig,
 @NotNull
 com.fishlib.configuration.Configuration configuration)
                 throws IOException

Deprecated.

use DataImportServer(DataImportServerLogFactory, DataImportServiceConfig, Configuration, SchemaService)

Delegate to DataImportServer(DataImportServerLogFactory, DataImportServiceConfig, Configuration, SchemaService) with the default schema service.

Parameters:

logNotUsed - not used - the non-deprecated method uses optional IrisLogCreator instead

disConfig - the parameters that define this DIS instance

configuration - the configuration to use when fetching settings that aren't included in disConfig

Throws:

IOException

Method Details

getTableDataService

public TableDataService getTableDataService()

Specified by:

getTableDataService in interface IDataImportServer

getSchemaService

public SchemaService getSchemaService()

Specified by:

getSchemaService in interface IDataImportServer

start

@Deprecated
public void start()
           throws IOException

Deprecated.

use startInWorker() instead

Throws:

IOException

startInWorker

public void startInWorker()
                   throws IOException

Start accepting connections in-worker.

Specified by:

startInWorker in interface IDataImportServer

Throws:

IOException

start

public void start(@NotNull
 com.fishlib.stats.StatsIntradayLogger statsIntradayLogger)
           throws IOException

Beginning tracking statistics and accepting connections.

Parameters:

statsIntradayLogger - the stats intraday logger

Throws:

IOException

shutdown

public void shutdown()

Specified by:

shutdown in interface IDataImportServer

Specified by:

shutdown in class com.fishlib.util.process.OnetimeShutdownTask

createSimpleProcessor

public SimpleDataImportStreamProcessor createSimpleProcessor(@NotNull
 String description,
 @NotNull
 FullTableLocationKey streamKey,
 @NotNull
 Runnable closeCallback,
 @NotNull
 SimpleDataImportStreamProcessor.OnReject onReject)

Create a SimpleDataImportStreamProcessor for importing data into this DataImportServer.

Specified by:

createSimpleProcessor in interface IDataImportServer

Parameters:

description - a description for the data connection

streamKey - destination information

closeCallback - callback to be invoked when the simple data stream is closed

onReject - callback to be invoked if the simple data stream is rejected

Returns:

a new SimpleDataImportStreamProcessor

main

public static void main(String... args)
                 throws IOException

Throws:

IOException

getDataImportServer

@Deprecated
public static DataImportServer getDataImportServer(com.fishlib.io.logger.Logger log,
 DataImportServiceConfig disConfig,
 com.fishlib.configuration.Configuration configuration,
 SchemaService schemaService)
                                            throws IOException

Deprecated.

Deprecated. Use getDataImportServer(DataImportServerLogFactory, DataImportServiceConfig, Configuration, SchemaService), passing a null IrisLogCreator if necessary.

Returns:

a new DataImportServer instance

Throws:

IOException

getDataImportServer

public static DataImportServer getDataImportServer(@Nullable
 DataImportServerLogFactory logCreator,
 DataImportServiceConfig disConfig,
 com.fishlib.configuration.Configuration configuration,
 SchemaService schemaService)
                                            throws IOException

Create a new DataImportServer instance, according to the configuration passed in.

Parameters:

logCreator - optional IrisLogCreator, will be used to create logger. If null, a global instance will be used, if available.

disConfig - the parameters that define this DIS instance

configuration - the configuration to use when fetching settings that aren't included in disConfig

schemaService - the SchemaService to use

Returns:

a new DataImportServer instance

Throws:

IOException