Package com.illumon.iris.importers

Class ColumnToolsImpl

java.lang.Object
com.illumon.iris.importers.ColumnToolsImpl
public class ColumnToolsImpl
extends Object
A class that provides methods for adding, removing, and modifying/rewriting historical Deephaven format column files.

  • Constructor Summary

    Constructors 
    Constructor Description
    ColumnToolsImpl​(com.fishlib.io.logger.Logger log, com.fishlib.util.process.FatalErrorReporter fatalErrorReporter, Database db)
    Creates an instance of the ColumnTools class, for the simple case with no progress callback and a thread pool size of 1.
    ColumnToolsImpl​(com.fishlib.io.logger.Logger log, com.fishlib.util.process.FatalErrorReporter fatalErrorReporter, Database db, StatusCallback statusCallback, int threadPoolSize)
    Creates an instance of the ColumnTools class, with options for a progress callback and specification of thread pool size.

  • Method Summary

    Modifier and Type Method Description
    void changeColumnType​(String namespace, String tableName, String columnName, Class<?> newDataType, String updateStatement)
    Changes the data type of a column for a table and updates the data for all historical partitions by applying the columnUpdateFormula to the source table.
    void deleteColumn​(String namespace, String tableName, String columnName)
    Deletes a column from a table and removes column files from historical partitions on disk.
    Table getSourceTable​(String namespace, String tableName, String partitioningColumnValue)  
    boolean isTableAllNative​(String namespace, String tableName)
    Checks whether all locations for a historical table are using native Deephaven formatting.
    void renameColumn​(String namespace, String tableName, String columnName, String newColumnName)
    Renames a column in a table and also renames corresponding column files in historical partitions on disk.
    void restartFinalization​(String namespace, String tableName)  
    void updateColumn​(String namespace, String tableName, String columnName, String updateStatement)
    Updates a column for all historical partitions by applying the columnUpdateFormula to the source table.
    void updateColumn​(String namespace, String tableName, String partitioningColumnValue, String columnName, Table sourceTable)
    Updates a column for a single historical partition by applying the columnUpdateFormula to the source table.
    void updateColumn​(String namespace, String tableName, String partitioningColumnValue, String columnName, String updateStatement)
    Updates a column for a single historical partition by applying the columnUpdateFormula to the source table.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • ColumnToolsImpl

      public ColumnToolsImpl​(@NotNull com.fishlib.io.logger.Logger log, @NotNull com.fishlib.util.process.FatalErrorReporter fatalErrorReporter, @NotNull Database db)
      Creates an instance of the ColumnTools class, for the simple case with no progress callback and a thread pool size of 1.
      Parameters:
      log - A logger to use for error and info logging.
      fatalErrorReporter - A reporter to use for fatal error reporting.
      db - A database instance to use for access to data from existing tables.

    • ColumnToolsImpl

      public ColumnToolsImpl​(@NotNull com.fishlib.io.logger.Logger log, @NotNull com.fishlib.util.process.FatalErrorReporter fatalErrorReporter, @NotNull Database db, @Nullable StatusCallback statusCallback, int threadPoolSize)
      Creates an instance of the ColumnTools class, with options for a progress callback and specification of thread pool size.
      Parameters:
      log - A logger to use for error and info logging.
      fatalErrorReporter - A reporter to use for fatal error reporting.
      db - A database instance to use for access to data from existing tables.
      statusCallback - An optional StatusCallback that can be used to update progress information.
      threadPoolSize - The number of threads to use for concurrent column file write operations. Maximum throughput will normally be with this number equal to the number of internal partitions in the historical storage.

  • Method Details

    • isTableAllNative

      public boolean isTableAllNative​(@NotNull String namespace, @NotNull String tableName)
      Checks whether all locations for a historical table are using native Deephaven formatting.
      Parameters:
      namespace - The namespace of the table to check
      tableName - The name of the table to check
      Returns:
      True if all locations for the table's historical data use Deephaven native format.

    • getSourceTable

      public Table getSourceTable​(@NotNull String namespace, @NotNull String tableName, @NotNull String partitioningColumnValue)

    • restartFinalization

      public void restartFinalization​(@NotNull String namespace, @NotNull String tableName)

    • deleteColumn

      public void deleteColumn​(@NotNull String namespace, @NotNull String tableName, @NotNull String columnName)
      Deletes a column from a table and removes column files from historical partitions on disk. The column to be deleted cannot be a partitioning or grouping column. This process will attempt to remove a matching column from the schema, and matching column files from disk even if no such column is found in the schema. This allows the procedure to be re-run if it fails or is interrupted, and also to be used to clean up column files when a column has already been removed from the schema through other methods.
      Parameters:
      namespace - The namespace containing the table to be updated
      tableName - The table to be updated
      columnName - The name of the column to delete

    • renameColumn

      public void renameColumn​(@NotNull String namespace, @NotNull String tableName, @NotNull String columnName, @NotNull String newColumnName)
      Renames a column in a table and also renames corresponding column files in historical partitions on disk. The column to be renamed cannot be a partitioning or grouping column.
      Parameters:
      namespace - The namespace containing the table to be updated
      tableName - The table to be updated
      columnName - The name of the column to rename
      newColumnName - The new name to apply to the column

    • changeColumnType

      public void changeColumnType​(@NotNull String namespace, @NotNull String tableName, @NotNull String columnName, @NotNull Class<?> newDataType, @NotNull String updateStatement)
      Changes the data type of a column for a table and updates the data for all historical partitions by applying the columnUpdateFormula to the source table. The column to be updated cannot be a partitioning or grouping column.
      Parameters:
      namespace - The namespace containing the table to be updated
      tableName - The table to be updated
      columnName - The name of the column to update
      newDataType - The class of the new type to apply to the column and corresponding schema
      updateStatement - The query language expression to apply to the source table to generate values for the column to be updated. This will be applied with .updateView(updateStatement) to the base table.

    • updateColumn

      public void updateColumn​(@NotNull String namespace, @NotNull String tableName, @NotNull String partitioningColumnValue, @NotNull String columnName, @NotNull Table sourceTable)
      Updates a column for a single historical partition by applying the columnUpdateFormula to the source table. The column to be updated cannot be a partitioning or grouping column.
      Parameters:
      namespace - The namespace containing the table to be updated
      tableName - The table to be updated
      partitioningColumnValue - The partitioning column value of the partition to be updated
      columnName - The name of the column to update
      sourceTable - A Table Based on one obtained from getSourceTable(String, String, String) which provides the updated data to be written to the specified column. The table passed as sourceTable must include the column indicated by columnName and the internal partition column that was in the original table returned from getSourceTable. sourceTable must also have the same number of rows, the same number of rows per internal partition, and an unchanged data type for the column to be updated.

    • updateColumn

      public void updateColumn​(@NotNull String namespace, @NotNull String tableName, @NotNull String partitioningColumnValue, @NotNull String columnName, @NotNull String updateStatement)
      Updates a column for a single historical partition by applying the columnUpdateFormula to the source table. The column to be updated cannot be a partitioning or grouping column.
      Parameters:
      namespace - The namespace containing the table to be updated
      tableName - The table to be updated
      partitioningColumnValue - The partitioning column value of the partition to be updated
      columnName - The name of the column to update
      updateStatement - The query language expression to apply to the source table to generate values for the column to be updated. This will be applied with .updateView(updateStatement) to the base table.

    • updateColumn

      public void updateColumn​(@NotNull String namespace, @NotNull String tableName, @NotNull String columnName, @NotNull String updateStatement)
      Updates a column for all historical partitions by applying the columnUpdateFormula to the source table. The column to be updated cannot be a partitioning or grouping column.
      Parameters:
      namespace - The namespace containing the table to be updated
      tableName - The table to be updated
      columnName - The name of the column to update
      updateStatement - The query language expression to apply to the source table to generate values for the column to be updated. This will be applied with .updateView(updateStatement) to the base table.