Package io.deephaven.importers.csv

Class CsvImporterHelperGeneral

java.lang.Object

io.deephaven.importers.csv.CsvImporterHelperGeneral

All Implemented Interfaces:

CsvImporterHelper, Closeable, AutoCloseable

public class CsvImporterHelperGeneral
extends Object
implements CsvImporterHelper

Implementation of CsvImporterHelper that handles general CSV import scenarios.

Method Summary

Modifier and Type	Method	Description
void	close()	Close the stream
int	getBufferSize()	Returns the buffer size that will be used when creating a FooterSkipBufferedReader
List<String>	getColumnNamesFromStream()	Get the list of column names from a CSV file; only call after it's been initialized with a stream
long	processImport(@NotNull Logger log, @NotNull ImportTableWriterFactory writerFactory, @NotNull Map<String,ImporterColumnDefinition> icdMap, @NotNull Map<String,String> importProperties, @NotNull String arrayDelimiter, @Nullable String constantColumnValue, @NotNull AtomicInteger errorCount, int maxError, boolean strict)	Process the source file or stream and persist to disk as a Table
void	setBufferSize(int bufferSize)	Sets the buffer size to use for a FooterSkipBufferedReader
void	validateImport()	Validate the import.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

getBufferSize

public int getBufferSize()

Description copied from interface: CsvImporterHelper

Returns the buffer size that will be used when creating a FooterSkipBufferedReader

Specified by:

getBufferSize in interface CsvImporterHelper

Returns:

int size of the buffer in characters

setBufferSize

public void setBufferSize(int bufferSize)

Description copied from interface: CsvImporterHelper

Sets the buffer size to use for a FooterSkipBufferedReader

Specified by:

setBufferSize in interface CsvImporterHelper

Parameters:

bufferSize - size of the buffer in characters

getColumnNamesFromStream

public List<String> getColumnNamesFromStream()

Description copied from interface: CsvImporterHelper

Get the list of column names from a CSV file; only call after it's been initialized with a stream

Specified by:

getColumnNamesFromStream in interface CsvImporterHelper

Returns:

the List of column names

processImport

public long processImport(@NotNull
 @NotNull Logger log,
 @NotNull
 @NotNull ImportTableWriterFactory writerFactory,
 @NotNull
 @NotNull Map<String,ImporterColumnDefinition> icdMap,
 @NotNull
 @NotNull Map<String,String> importProperties,
 @NotNull
 @NotNull String arrayDelimiter,
 @Nullable
 @Nullable String constantColumnValue,
 @NotNull
 @NotNull AtomicInteger errorCount,
 int maxError,
 boolean strict)
                   throws IOException

Description copied from interface: CsvImporterHelper

Process the source file or stream and persist to disk as a Table

Specified by:

processImport in interface CsvImporterHelper

Parameters:

log - The passed-down logger

writerFactory - The passed down ImportTableWriterFactory

icdMap - The column name to ImporterColumnDefinition map

importProperties - Provides basic import attributes

arrayDelimiter - Delimiter used to parse array data types

constantColumnValue - A String to materialize as the source column when an ImportColumn is defined with a sourceType of CONSTANT (aka ImporterColumnDefinition$IrisImportConstant). Can be null.

errorCount - Holds a record of parse errors

maxError - Maximum number of field conversion failures allowed

strict - Whether to fail if a field fails conversion

Returns:

The number of rows processed

Throws:

IOException - from the import

close

public void close()
           throws IOException

Description copied from interface: CsvImporterHelper

Close the stream

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface CsvImporterHelper

Throws:

IOException - if an error occurs

validateImport

public void validateImport()
                    throws ImportException

Description copied from interface: CsvImporterHelper

Validate the import.

Specified by:

validateImport in interface CsvImporterHelper

Throws:

ImportException - if the import failed