Package com.illumon.iris.importers.csv.parsers

Class CsvObjectParserBase<TYPE,TARRAY>

java.lang.Object
com.illumon.iris.importers.csv.parsers.CsvObjectParserBase<TYPE,TARRAY>
Type Parameters:
TYPE - The column data type
TARRAY - The stored values array data type (for example Integer DataType column would have int[] array data type)
All Implemented Interfaces:
io.deephaven.csv.parsers.Parser<TARRAY>
public abstract class CsvObjectParserBase<TYPE,TARRAY> extends Object
This is base class for custom parsers that have a non-primitive data type in general.

  • Nested Class Summary

    Nested classes/interfaces inherited from interface io.deephaven.csv.parsers.Parser

    io.deephaven.csv.parsers.Parser.GlobalContext, io.deephaven.csv.parsers.Parser.ParserContext<TARRAY extends Object>

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected final boolean
    applyTransform
     
    protected final AppendableColumnSink<TYPE,TARRAY>
    columnSink
     
    protected final ImportColumnDataTransformer
    dataTransformer
     
    protected final io.deephaven.csv.util.MutableObject<TYPE>
    holder
     
    protected static final int
    INITIAL_TRANSFORM_BUFFER_SIZE
     
    protected final CsvParserContext
    parserContext
     
    protected char[]
    transformBuffer
     

    Fields inherited from interface io.deephaven.csv.parsers.Parser

    CHUNK_SIZE

  • Constructor Summary

    Constructors
    Modifier
    Constructor
    Description
    protected
    CsvObjectParserBase(AppendableColumnSink<TYPE,TARRAY> columnSink, CsvParserContext parserContext, String dataType, boolean failDefault)
    Base Array / Object Parser class that supports error count tracking.

  • Method Summary

    Modifier and Type
    Method
    Description
    protected abstract TYPE
    getCurrentValue(String bufferValue)
    Returns the parsed value of the current byte slice that has been passed in as a String value.
    protected abstract int
    getLength(TARRAY values)
    Convenience method that returns the length of the parameter values which is the array returned for Parser.ParserContext.valueChunk()
    protected char[]
    getTransformData(com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih)
    Method to update reusable transform buffer
    protected boolean
    suppressErrorAndUpdateWithDefault(long rowNum, boolean isNull, TARRAY values, int chunkIndex)
    On null or error conditions verifies if allowed error count is exhausted by considering strict and failDefault attributes.
    protected long
    tryParse(io.deephaven.csv.parsers.Parser.GlobalContext gctx, io.deephaven.csv.parsers.Parser.ParserContext<TARRAY> pctx, com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih, long begin, long end, boolean appending)
    The method is similar in behavior to See Parser.tryParse(GlobalContext, ParserContext, IteratorHolder, long, long, boolean).
    long
    tryParse(io.deephaven.csv.parsers.Parser.GlobalContext gctx, io.deephaven.csv.parsers.Parser.ParserContext<TARRAY> pctx, io.deephaven.csv.parsers.IteratorHolder ih, long begin, long end, boolean appending)
    Redirects to invoke tryParse with a ColumnDataIteratorHolder Implementation.
    protected abstract boolean
    tryUpdateWithDefault(TARRAY values, int chunkIndex)
    Implementations should update with default value at the current position.
    protected abstract void
    updateCurrentValue(TARRAY values, TYPE currentValue, int chunkIndex)
    Convenience method that updates the values parameter which is an array with the given current value at the given chunkIndex.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    Methods inherited from interface io.deephaven.csv.parsers.Parser

    makeParserContext

  • Field Details

    • holder

      protected final io.deephaven.csv.util.MutableObject<TYPE> holder

    • INITIAL_TRANSFORM_BUFFER_SIZE

      protected static final int INITIAL_TRANSFORM_BUFFER_SIZE
      See Also:

    • columnSink

      protected final AppendableColumnSink<TYPE,TARRAY> columnSink

    • dataTransformer

      protected final ImportColumnDataTransformer dataTransformer

    • parserContext

      protected final CsvParserContext parserContext

    • applyTransform

      protected final boolean applyTransform

    • transformBuffer

      protected char[] transformBuffer

  • Constructor Details

    • CsvObjectParserBase

      protected CsvObjectParserBase(@NotNull AppendableColumnSink<TYPE,TARRAY> columnSink, @NotNull CsvParserContext parserContext, @NotNull String dataType, boolean failDefault)
      Base Array / Object Parser class that supports error count tracking.
      Parameters:
      columnSink - Appendable column sink for which the parser is used
      parserContext - The context object that holds common attributes passed down and needed by all Custom Parsers
      dataType - Data type supported by parser to be used in log or error message
      failDefault - Whether (true) to use the default value (onEmpty) if parsing/conversion of the source value fails

  • Method Details

    • tryParse

      protected long tryParse(@NotNull io.deephaven.csv.parsers.Parser.GlobalContext gctx, @NotNull io.deephaven.csv.parsers.Parser.ParserContext<TARRAY> pctx, @NotNull com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih, long begin, long end, boolean appending) throws io.deephaven.csv.util.CsvReaderException
      The method is similar in behavior to See Parser.tryParse(GlobalContext, ParserContext, IteratorHolder, long, long, boolean). Where it differs is the method is invoked from the tryParse method of the SinkHolderParser, which controls the traversal of the original IteratorHolder. Below is more information on how the tryParse is designed to work

      Tries to parse the data pointed to by SinkHolderIterator 'ih' into a Sink. The method parses as many values as it can. It stops when: 1. The range [destBegin,destEnd) is full, or 2. The iterator ih is exhausted, or 3. The code encounters a source value that it is unable to parse.

      Parameters:
      gctx - The Parser.GlobalContext holding various shared parameters for the parse. This will be shared among parsers of different types as the type inference process proceeds.
      pctx - The Parser.ParserContext for this specific parser. It will be the object created by the call to {Parser#makeContext}. If the caller calls tryParse multiple times (for example during two-phase parsing), it will pass the same Parser.ParserContext object each time.
      ih - An ColumnDataIteratorHolder pointing to the data. It is already pointing to the current element or the end (in other words, it has had SinkHolderIterator.tryMoveNext called on it at least once). The reason for this invariant is that other code (controlling logic and other parsers in the sink holder) have needed to peek at the current element before getting here in order to decide what to do.
      begin - The start of the range (inclusive) to write values to.
      end - The end of the range (exclusive) to write values to. This can also be a very large value like Long.MAX_VALUE if the caller does not know how many values there are.
      appending - Whether the parser is being called in a mode where it is appending to the end of the Sink or replacing previously-written pad values in the Sink. This value is simply passed on to Sink.write which may use it as a hint to slightly simplify its logic.
      Returns:
      The end range (exclusive) of the values parsed. Returns begin if no values were parsed.
      Throws:
      io.deephaven.csv.util.CsvReaderException - when an exception happens in the implementation

    • getCurrentValue

      @Nullable protected abstract TYPE getCurrentValue(@NotNull String bufferValue) throws InputMismatchException, NumberFormatException
      Returns the parsed value of the current byte slice that has been passed in as a String value.
      Parameters:
      bufferValue - The current String value of the current byte slice
      Returns:
      the parsed value of the current byte slice that has been passed in as a String value.
      Throws:
      InputMismatchException - thrown while retrieving the value
      NumberFormatException - thrown in case of Number data types when parsing fails due to number format errors.

    • updateCurrentValue

      protected abstract void updateCurrentValue(@NotNull TARRAY values, @Nullable TYPE currentValue, int chunkIndex)
      Convenience method that updates the values parameter which is an array with the given current value at the given chunkIndex.
      Parameters:
      values - the values array retrieved using Parser.ParserContext.valueChunk()
      currentValue - the value that should be updated in the values array at the given index
      chunkIndex - the index where the currentValue needs to be updated

    • getLength

      protected abstract int getLength(@NotNull TARRAY values)
      Convenience method that returns the length of the parameter values which is the array returned for Parser.ParserContext.valueChunk()
      Parameters:
      values - the values array retrieved using Parser.ParserContext.valueChunk()
      Returns:
      the length of the array

    • getTransformData

      @NotNull protected char[] getTransformData(@NotNull com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih)
      Method to update reusable transform buffer

    • suppressErrorAndUpdateWithDefault

      protected boolean suppressErrorAndUpdateWithDefault(long rowNum, boolean isNull, @NotNull TARRAY values, int chunkIndex) throws io.deephaven.csv.util.CsvReaderException
      On null or error conditions verifies if allowed error count is exhausted by considering strict and failDefault attributes. Updates with default value if applicable.
      Parameters:
      rowNum - The current row for entire import
      isNull - Indicates if this was invoked for a null value cell
      values - The current values buffer
      chunkIndex - The current position to update on values buffer
      Returns:
      true if successfully updated with supplied default value
      Throws:
      io.deephaven.csv.util.CsvReaderException - when exceeds max error count.

    • tryUpdateWithDefault

      protected abstract boolean tryUpdateWithDefault(@NotNull TARRAY values, int chunkIndex)
      Implementations should update with default value at the current position.
      Parameters:
      values - The current values buffer
      chunkIndex - The current position to update on values buffer
      Returns:
      true if successfully updated with supplied default value

    • tryParse

      public long tryParse(@NotNull io.deephaven.csv.parsers.Parser.GlobalContext gctx, @NotNull io.deephaven.csv.parsers.Parser.ParserContext<TARRAY> pctx, @Nullable io.deephaven.csv.parsers.IteratorHolder ih, long begin, long end, boolean appending) throws io.deephaven.csv.util.CsvReaderException
      Redirects to invoke tryParse with a ColumnDataIteratorHolder Implementation. See Parser.tryParse(GlobalContext, ParserContext, IteratorHolder, long, long, boolean)
      Specified by:
      tryParse in interface io.deephaven.csv.parsers.Parser<TYPE>
      Throws:
      io.deephaven.csv.util.CsvReaderException