Package com.illumon.iris.importers.csv

Class CsvImportTools

java.lang.Object

com.illumon.iris.importers.csv.CsvImportTools

public class CsvImportTools
extends Object

Util class that provides ability to import csv using dhc-fast-csv parser.

Constructor Summary

Constructors

Constructor	Description
CsvImportTools()

Method Summary

Modifier and Type	Method	Description
static long	importCsv(InputStream stream, io.deephaven.csv.CsvSpecs.Builder specBuilder, ImportTableWriterFactory tableWriterFactory, com.fishlib.io.logger.Logger log, List<String> columnNamesInFile, Map<String,ImporterColumnDefinition> icdMap, Map<String,String> importProperties, AtomicInteger errorCount, String arrayDelimiter, String constantColumnValue, String currentPartition, int maxError, boolean strict, boolean fromSplitFile)	Imports and writes the csv data to disk.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

CsvImportTools

public CsvImportTools()

Method Details

importCsv

public static long importCsv(@NotNull
 InputStream stream,
 @NotNull
 io.deephaven.csv.CsvSpecs.Builder specBuilder,
 @NotNull
 ImportTableWriterFactory tableWriterFactory,
 @NotNull
 com.fishlib.io.logger.Logger log,
 @NotNull
 List<String> columnNamesInFile,
 @NotNull
 Map<String,ImporterColumnDefinition> icdMap,
 @NotNull
 Map<String,String> importProperties,
 @NotNull
 AtomicInteger errorCount,
 @NotNull
 String arrayDelimiter,
 @Nullable
 String constantColumnValue,
 @Nullable
 String currentPartition,
 int maxError,
 boolean strict,
 boolean fromSplitFile)
                      throws IOException

Imports and writes the csv data to disk.

Parameters:

stream - InputStream from which to read CSV data

specBuilder - The CsvSpecs.Builder

tableWriterFactory - The passed down ImportTableWriterFactory

log - The passed-down logger

columnNamesInFile - The column headers in source

icdMap - The column name to ImporterColumnDefinition map

importProperties - Provides basic import attributes

errorCount - Holds the current error count across all parsers being used to import csv

arrayDelimiter - Delimiter used to parse array data types

constantColumnValue - A String to materialize as the source column when an ImportColumn is defined with a sourceType of CONSTANT (aka ImporterColumnDefinition$IrisImportConstant). Can be null.

currentPartition - The partition value if coming from split File

maxError - Maximum number of field conversion failures allowed

strict - Whether to fail if a field fails conversion

fromSplitFile - Indicates if invoked from splitFile (the significance there will be only one partition)

Returns:

Returns the number of rows processed

Throws:

IOException - thrown when encountering an error in the import call