Package com.illumon.iris.importers.merge

Class MergeDestination

java.lang.Object
com.illumon.iris.importers.merge.MergeDestination
All Implemented Interfaces:
com.fishlib.base.log.LogOutputAppendable
public class MergeDestination
extends Object
implements com.fishlib.base.log.LogOutputAppendable
Represents an output location for the merge operation.

  • Constructor Details

    • MergeDestination

      public MergeDestination​(@NotNull File writableInternalPartition, @NotNull String partitioningColumnValue, @NotNull String tableName, @NotNull String operationName)
      For internal Deephaven use only.

  • Method Details

    • permanentDirectoryExists

      public boolean permanentDirectoryExists()
      For internal Deephaven use only.

    • getTemporaryDirectory

      public File getTemporaryDirectory()
      For internal Deephaven use only.

    • cleanup

      public void cleanup()
      For internal Deephaven use only.

    • setInputSubTable

      public void setInputSubTable​(@NotNull Table inputSubTable)
      For internal Deephaven use only.

    • groupInputSubTable

      public void groupInputSubTable​(@NotNull com.fishlib.io.logger.Logger log, @NotNull String[] groupingColumnNames)
      For internal Deephaven use only.

    • initializeOutputTable

      public void initializeOutputTable​(@NotNull TableDefinition writableTableDefinition)
      For internal Deephaven use only.

    • computeGroupOrder

      public void computeGroupOrder​(@NotNull com.fishlib.io.logger.Logger log, @NotNull String[] groupingColumnNames)
      For internal Deephaven use only.

    • writeGroupingColumns

      public void writeGroupingColumns​(@NotNull com.fishlib.io.logger.Logger log, @NotNull com.fishlib.util.process.FatalErrorReporter fatalErrorReporter, SyncOperator syncOperator)
      Write the grouping columns for Deephaven-format merges.

    • writeGroupingIndex

      public void writeGroupingIndex​(com.fishlib.io.logger.Logger log, TableDefinition writableTableDefinition)
      Write out the grouping index table. The table consists of the grouping column values and a column containing the index.
      Parameters:
      log - the logger for output
      writableTableDefinition - the original table definition to derive the index table definition from.

    • extractGroupings

      public Map<String,​Map<?,​ReadOnlyIndex>> extractGroupings​(@NotNull com.fishlib.io.logger.Logger log)
      Extract the grouping columns for Parquet-format merges.

    • writeOneColumn

      public void writeOneColumn​(@NotNull com.fishlib.io.logger.Logger log, @NotNull com.fishlib.util.process.FatalErrorReporter fatalErrorReporter, @NotNull SyncOperator syncOperator, @NotNull ColumnSource columnSource, @NotNull String columnName, @NotNull Index sourceTableIndex)
      For internal Deephaven use only.

    • writeColumn

      public void writeColumn​(@NotNull com.fishlib.io.logger.Logger log, @NotNull com.fishlib.util.process.FatalErrorReporter fatalErrorReporter, @NotNull SyncOperator syncOperator, @NotNull ColumnSource<?> columnSource, @NotNull String columnName)
      For internal Deephaven use only.

    • writeMetadata

      public void writeMetadata​(@NotNull com.fishlib.io.logger.Logger log)
      For internal Deephaven use only.

    • computeSortedOrderDeephaven

      public void computeSortedOrderDeephaven​(@NotNull com.fishlib.io.logger.Logger log, @Nullable String sortColumnFormula, @Nullable com.illumon.dataobjects.ColumnDefinition<?> sortColumnDefinition, List<String[]> indexDataSets)
      For Deephaven format, compute the sorted order and set the destination's subTableCopiers.

    • computeSortedOrderParquet

      public ParquetTableWriter.DestinationInfo computeSortedOrderParquet​(@NotNull com.fishlib.io.logger.Logger log, @NotNull String logPrefix, @Nullable String sortColumnFormula, @Nullable com.illumon.dataobjects.ColumnDefinition<?> sortColumnDefinition, @NotNull Map<String,​Map<?,​ReadOnlyIndex>> groupings)
      For Parquet format, compute the sorted order and return a DestinationInfo.

    • computeDataIndexes

      public void computeDataIndexes​(com.fishlib.io.logger.Logger log)
      For internal Deephaven use only.

    • install

      public void install()
      For internal Deephaven use only.

    • getDestinationOperationName

      public String getDestinationOperationName()
      For internal Deephaven use only.

    • getInternalPartitionName

      public String getInternalPartitionName()
      For internal Deephaven use only.

    • getLogPrefix

      public String getLogPrefix()
      For internal Deephaven use only.

    • getPermanentDirectory

      public File getPermanentDirectory()
      For internal Deephaven use only.

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • append

      public com.fishlib.base.log.LogOutput append​(com.fishlib.base.log.LogOutput logOutput)
      Specified by:
      append in interface com.fishlib.base.log.LogOutputAppendable