Package com.illumon.iris.importers.csv.sink

Interface AppendableColumnSink<DATA_TYPE,​TARRAY>

All Superinterfaces:
AppendableSink<DATA_TYPE,​TARRAY>, PartitionUpdatesObserver, io.deephaven.csv.sinks.Sink<TARRAY>
All Known Implementing Classes:
BaseAppendableColumnSink
public interface AppendableColumnSink<DATA_TYPE,​TARRAY>
extends AppendableSink<DATA_TYPE,​TARRAY>
The interface encapsulates the table column properties and provides wrapper methods to invoke persist calls on underlying LocalAppendableColumn

  • Method Details

    • getColumnName

      String getColumnName()
      Getter for column name

    • getColumnDataTransformer

      ImportColumnDataTransformer getColumnDataTransformer()
      Getter for ColumnDataTransformer

    • supportsTransformations

      boolean supportsTransformations()
      Returns true if the ColumnDataTransformer supports transformation

    • isPartitionCol

      boolean isPartitionCol()
      Returns true if the column is defined as a partition column in the schema

    • isColumnInSource

      boolean isColumnInSource()
      Returns true if the column is present in the source csv file. This is determined by whether the column name exists in the columnNamesInFile attribute of CsvImportHelper

    • isColumnInSchema

      boolean isColumnInSchema()
      Returns true if the column is present in schema associated with the csv import

    • isConstantColumn

      boolean isConstantColumn()
      Returns true if the ColumnDataTransformer has the hasConstant attribute return true

    • publishRowUpdate

      void publishRowUpdate​(int size, long end)
      Publish row updates to RowUpdateObserver.

    • isColumnOnlyInSchema

      default boolean isColumnOnlyInSchema()
      If the column is defined in the schema but not in the source csv file (Constant columns are an example for such columns)

    • addAppendableColumn

      void addAppendableColumn​(String partition, LocalAppendableColumn<DATA_TYPE> appendableColumn)
      The method provides the partition and its associated LocalAppendableColumn This information should be cached in the column sink implementation and should be used when updating the column values based on PartitionParserUpdate that will be pushed to the column when the table has a partitioning column and the values of the column should be persisted in the appropriate partition.
      Parameters:
      partition - Partition value as a string
      appendableColumn - The LocalAppendableColumn of the column associated for this partition

    • evict

      void evict​(String partition)
      The provided partition should be evicted from the current partition column cache. This should be registered as one of the evicted partitions.
      Parameters:
      partition - Partition value as a string, that should be evicted from the local column cache of partitions

    • addBytes

      default void addBytes​(@NotNull byte[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the LocalAppendableColumn.addBytes(byte[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • addBooleans

      default void addBooleans​(@NotNull Boolean[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the BaseAppendableColumn.addBooleans(Boolean[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • addChars

      default void addChars​(@NotNull char[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the LocalAppendableColumn.addChars(char[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • addDoubles

      default void addDoubles​(@NotNull double[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the LocalAppendableColumn.addDouble(double, long) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • addFloats

      default void addFloats​(@NotNull float[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the LocalAppendableColumn.addFloats(float[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • addInts

      default void addInts​(@NotNull int[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the LocalAppendableColumn.addInts(int[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • addLongs

      default void addLongs​(@NotNull long[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the LocalAppendableColumn.addLongs(long[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • addShorts

      default void addShorts​(@NotNull short[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the LocalAppendableColumn.addShorts(short[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value

    • add

      default void add​(@NotNull DATA_TYPE[] values, int startIndex, int length, long destEnd, boolean isSingleValue)
      The wrapper method around the BaseAppendableColumn.add(Object[], int, int) The method should make sure the received values are persisted to the right partitions when the table schema supports partitions. To allow for this, the persisting of values may be blocked until the partition column is processed for the same chunk. In addition, if other columns have registered an interest in receiving current chunk details then that information should be published as the first operation.
      Parameters:
      values - The values to be persisted in the chunk
      startIndex - The start index at which to persist
      length - The length of the values that should be persisted as part of this chunk
      destEnd - The total no of rows processed up to the end of this chunk
      isSingleValue - If all the values for the update are the same constant value