Package com.illumon.iris.importers.csv.parsers

Class CsvPartitionColumnParser

java.lang.Object
com.illumon.iris.importers.csv.parsers.CsvPartitionColumnParser
All Implemented Interfaces:
io.deephaven.csv.parsers.Parser<String[]>
public final class CsvPartitionColumnParser extends Object
The parser will be used for the partition column when the schema has a partition column defined. The primary partition observer will listen to the Parser updates and will publish the parser updates to other columns in table. Note the parser will not write the updates to a Sink (A partition column will not be persisted with the other columns in the table).

  • Nested Class Summary

    Nested classes/interfaces inherited from interface io.deephaven.csv.parsers.Parser

    io.deephaven.csv.parsers.Parser.GlobalContext, io.deephaven.csv.parsers.Parser.ParserContext<TARRAY extends Object>

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    protected static final int
    INITIAL_TRANSFORM_BUFFER_SIZE
     
    protected final CsvParserContext
    parserContext
     
    protected char[]
    transformBuffer
     

    Fields inherited from interface io.deephaven.csv.parsers.Parser

    CHUNK_SIZE

  • Method Summary

    Modifier and Type
    Method
    Description
    protected char[]
    getTransformData(com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih)
    Method to update reusable transform buffer
    static CsvPartitionColumnParser
    make(PartitionUpdatesObserver primaryObserver, CsvParserContext parserContext, AppendableColumnSink<String,String[]> columnWrapper)
    Used to create the partition column parser, when the schema has a partition column.
    io.deephaven.csv.parsers.Parser.ParserContext<String[]>
    makeParserContext(io.deephaven.csv.parsers.Parser.GlobalContext gctx, int chunkSize)
     
    void
    parseChunkUpdate(int size, long destEnd)
    The chunk updates are received by registering to AppendableColumn in the source.
    protected boolean
    suppressErrorAndUpdateWithDefault(long rowNum, boolean isNull, String[] values, int chunkIndex)
    On null or error conditions verifies if allowed error count is exhausted by considering strict and failDefault attributes.
    protected long
    tryParse(io.deephaven.csv.parsers.Parser.GlobalContext gctx, io.deephaven.csv.parsers.Parser.ParserContext<String[]> pctx, com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih, long begin, long end, boolean appending)
    The method is similar in behavior to See Parser.tryParse(GlobalContext, ParserContext, IteratorHolder, long, long, boolean).
    long
    tryParse(io.deephaven.csv.parsers.Parser.GlobalContext gctx, io.deephaven.csv.parsers.Parser.ParserContext<String[]> pctx, io.deephaven.csv.parsers.IteratorHolder ih, long begin, long end, boolean appending)
    Redirects to invoke tryParse with a ColumnDataIteratorHolder Implementation.
    protected boolean
    tryUpdateWithDefault(String[] values, int chunkIndex)
    Implementations should update with default value at the current position.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Field Details

    • INITIAL_TRANSFORM_BUFFER_SIZE

      protected static final int INITIAL_TRANSFORM_BUFFER_SIZE
      See Also:

    • parserContext

      protected final CsvParserContext parserContext

    • transformBuffer

      protected char[] transformBuffer

  • Method Details

    • make

      public static CsvPartitionColumnParser make(@NotNull PartitionUpdatesObserver primaryObserver, @NotNull CsvParserContext parserContext, @NotNull AppendableColumnSink<String,String[]> columnWrapper)
      Used to create the partition column parser, when the schema has a partition column.
      Parameters:
      primaryObserver - The observer that would observe the parsers partition update, and then delegate as it deems necessary.
      parserContext - The context object that holds common attributes passed down and needed by all Custom Parsers
      columnWrapper - The partition column sink
      Returns:
      The PartitionColumnParser.

    • makeParserContext

      @NotNull public io.deephaven.csv.parsers.Parser.ParserContext<String[]> makeParserContext(io.deephaven.csv.parsers.Parser.GlobalContext gctx, int chunkSize)

    • tryParse

      protected long tryParse(@NotNull io.deephaven.csv.parsers.Parser.GlobalContext gctx, @NotNull io.deephaven.csv.parsers.Parser.ParserContext<String[]> pctx, @NotNull com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih, long begin, long end, boolean appending) throws io.deephaven.csv.util.CsvReaderException
      The method is similar in behavior to See Parser.tryParse(GlobalContext, ParserContext, IteratorHolder, long, long, boolean). Where it differs is the method is invoked from the tryParse method of the SinkHolderParser, which controls the traversal of the original IteratorHolder. Below is more information on how the tryParse is designed to work

      Tries to parse the data pointed to by SinkHolderIterator 'ih' into a Sink. The method parses as many values as it can. It stops when: 1. The range [destBegin,destEnd) is full, or 2. The iterator ih is exhausted, or 3. The code encounters a source value that it is unable to parse.

      Parameters:
      gctx - The Parser.GlobalContext holding various shared parameters for the parse. This will be shared among parsers of different types as the type inference process proceeds.
      pctx - The Parser.ParserContext for this specific parser. It will be the object created by the call to {Parser#makeContext}. If the caller calls tryParse multiple times (for example during two-phase parsing), it will pass the same Parser.ParserContext object each time.
      ih - An ColumnDataIteratorHolder pointing to the data. It is already pointing to the current element or the end (in other words, it has had SinkHolderIterator.tryMoveNext called on it at least once). The reason for this invariant is that other code (controlling logic and other parsers in the sink holder) have needed to peek at the current element before getting here in order to decide what to do.
      begin - The start of the range (inclusive) to write values to.
      end - The end of the range (exclusive) to write values to. This can also be a very large value like Long.MAX_VALUE if the caller does not know how many values there are.
      appending - Whether the parser is being called in a mode where it is appending to the end of the Sink or replacing previously-written pad values in the Sink. This value is simply passed on to Sink.write which may use it as a hint to slightly simplify its logic.
      Returns:
      The end range (exclusive) of the values parsed. Returns begin if no values were parsed.
      Throws:
      io.deephaven.csv.util.CsvReaderException - when an exception happens in the implementation

    • parseChunkUpdate

      public void parseChunkUpdate(int size, long destEnd)
      The chunk updates are received by registering to AppendableColumn in the source. This approach should be used when the partition column is not in the source file.
      Parameters:
      size - The length of the update
      destEnd - The passed-down destination End parameter of the update

    • tryUpdateWithDefault

      protected boolean tryUpdateWithDefault(@NotNull String[] values, int chunkIndex)
      Implementations should update with default value at the current position.
      Parameters:
      values - The current values buffer
      chunkIndex - The current position to update on values buffer
      Returns:
      true if successfully updated with supplied default value

    • getTransformData

      @NotNull protected char[] getTransformData(@NotNull com.illumon.iris.importers.csv.parsers.ColumnDataIteratorHolder ih)
      Method to update reusable transform buffer

    • suppressErrorAndUpdateWithDefault

      protected boolean suppressErrorAndUpdateWithDefault(long rowNum, boolean isNull, @NotNull String[] values, int chunkIndex) throws io.deephaven.csv.util.CsvReaderException
      On null or error conditions verifies if allowed error count is exhausted by considering strict and failDefault attributes. Updates with default value if applicable.
      Parameters:
      rowNum - The current row for entire import
      isNull - Indicates if this was invoked for a null value cell
      values - The current values buffer
      chunkIndex - The current position to update on values buffer
      Returns:
      true if successfully updated with supplied default value
      Throws:
      io.deephaven.csv.util.CsvReaderException - when exceeds max error count.

    • tryParse

      public long tryParse(@NotNull io.deephaven.csv.parsers.Parser.GlobalContext gctx, @NotNull io.deephaven.csv.parsers.Parser.ParserContext<String[]> pctx, @Nullable io.deephaven.csv.parsers.IteratorHolder ih, long begin, long end, boolean appending) throws io.deephaven.csv.util.CsvReaderException
      Redirects to invoke tryParse with a ColumnDataIteratorHolder Implementation. See Parser.tryParse(GlobalContext, ParserContext, IteratorHolder, long, long, boolean)
      Specified by:
      tryParse in interface io.deephaven.csv.parsers.Parser<TYPE>
      Throws:
      io.deephaven.csv.util.CsvReaderException