Package io.deephaven.importers

Class GeneralImporter<T extends ImportFieldWriter>

java.lang.Object

io.deephaven.importers.GeneralImporter<T>

Type Parameters:

T - the type of ImportFieldWriter used by this importer

Direct Known Subclasses:

GeneralCsvImporter, GeneralJdbcImporter

public abstract class GeneralImporter<T extends ImportFieldWriter>
extends Object

Abstract class providing methods used when importing data to be written to Deephaven columns.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	GeneralImporter.CacheEntry	This class is used to hold a TableWriter and its associated FieldWriters.
class	GeneralImporter.IntradayPartitionSupplier<Context>

Field Summary

Fields

Modifier and Type	Field	Description
protected final Map<String,String>	customImportProperties
protected final ImportTableWriterFactory	importTableWriterFactory
protected final String	intradayPartitionColumn
protected final Logger	log
protected final boolean	strict

Constructor Summary

Constructors

Constructor	Description
GeneralImporter(@NotNull Logger log, @NotNull ImportTableWriterFactory importTableWriterFactory, @Nullable String intradayPartitionColumn, boolean strict, @Nullable Map<String,String> customImportProperties)

Method Summary

Modifier and Type	Method	Description
abstract T	buildEndOfRecordFieldWriter(io.deephaven.shadow.enterprise.com.illumon.iris.binarystore.TableWriter tableWriter)
abstract Function<io.deephaven.shadow.enterprise.com.illumon.iris.binarystore.RowSetter,T>	buildFieldWriterFactory(ImporterColumnDefinition name, Class setterType, Map<String,String> importProperties, String actualPartition)
void	closeAllWriters()
abstract Map<String,ImporterColumnDefinition>	getColumnDefinitions()
ImportTableWriterFactory	getImportTableWriterFactory()	Get the ImportTableWriterFactory.
static int	getMaxOpenTableWriters()
protected GeneralImporter<T>.CacheEntry	getTableEntry(String partition, List<String> columns, Map<String,String> importProperties)	Get the GeneralImporter<T extends ImportFieldWriter>.CacheEntry for the specified partition, creating a new one if necessary.
static void	setMaxOpenTableWriters(int maxWriters)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

log

protected final Logger log

importTableWriterFactory

protected final ImportTableWriterFactory importTableWriterFactory

intradayPartitionColumn

protected final String intradayPartitionColumn

strict

protected final boolean strict

customImportProperties

protected final Map<String,String> customImportProperties

Constructor Details

GeneralImporter

public GeneralImporter(@NotNull
 @NotNull Logger log,
 @NotNull
 @NotNull ImportTableWriterFactory importTableWriterFactory,
 @Nullable
 @Nullable String intradayPartitionColumn,
 boolean strict,
 @Nullable
 @Nullable Map<String,String> customImportProperties)

Method Details

getTableEntry

protected GeneralImporter<T>.CacheEntry getTableEntry(String partition,
 List<String> columns,
 Map<String,String> importProperties)

Get the GeneralImporter<T extends ImportFieldWriter>.CacheEntry for the specified partition, creating a new one if necessary.

Parameters:

partition - the partition value

columns - the list of source columns

importProperties - the import properties

Returns:

the GeneralImporter<T extends ImportFieldWriter>.CacheEntry

closeAllWriters

public void closeAllWriters()
                     throws IOException

Throws:

IOException - from the close calls

getColumnDefinitions

public abstract Map<String,ImporterColumnDefinition> getColumnDefinitions()

buildFieldWriterFactory

public abstract Function<io.deephaven.shadow.enterprise.com.illumon.iris.binarystore.RowSetter,T> buildFieldWriterFactory(ImporterColumnDefinition name,
 Class setterType,
 Map<String,String> importProperties,
 String actualPartition)

buildEndOfRecordFieldWriter

public abstract T buildEndOfRecordFieldWriter(io.deephaven.shadow.enterprise.com.illumon.iris.binarystore.TableWriter tableWriter)

getMaxOpenTableWriters

public static int getMaxOpenTableWriters()

setMaxOpenTableWriters

@TestUseOnly
public static void setMaxOpenTableWriters(int maxWriters)

getImportTableWriterFactory

public ImportTableWriterFactory getImportTableWriterFactory()

Get the ImportTableWriterFactory.

Returns:

the import table writer factory