Package com.illumon.iris.importers.util

Interface MergeParameters

@Immutable public interface MergeParameters
A set of parameters to be used while performing a merge with MergeData. Use the builder() method to construct a builder to use to make a concrete object.

  • Field Details

  • Method Details

    • builder

      static MergeParameters.Builder builder()
      Construct an empty builder.
      Returns:
      a new empty MergeParameters.Builder

    • builder

      static MergeParameters.Builder builder(@NotNull Database db, @NotNull String namespace, @NotNull String tableName)
      Construct a builder, pre-populating the database namespace and table name properties.
      Parameters:
      db - the database
      namespace - the namespace to merge to
      tableName - the table name to merge to
      Returns:
      a MergeParameters.Builder populated with the specified parameters/

    • builder

      static MergeParameters.Builder builder(@NotNull com.fishlib.io.logger.Logger log, @NotNull Database db, @NotNull String namespace, @NotNull String tableName)
      Construct a builder, pre-populating the logger, database namespace and table name properties.
      Parameters:
      log - the logger
      db - the database
      namespace - the namespace to merge to
      tableName - the table name to merge to
      Returns:
      a MergeParameters.Builder populated with the specified parameters/

    • logger

      @Default @Nullable default com.fishlib.io.logger.Logger logger()
      Get the logger to use.
      Returns:
      the logger

    • db

      @Default @Nullable default Database db()
      Get the database to fetch tables from.
      Returns:
      the database

    • tdsName

      @Default @Nullable default String tdsName()
      Get the name of the Table data service to use.
      Returns:
      the table data service name

    • namespace

      String namespace()
      Get the namespace to merge data into.
      Returns:
      the namespace

    • tableName

      String tableName()
      Get the Table to merge into
      Returns:
      the table name to merge into

    • partitionColumnValue

      @Default @Nullable default String partitionColumnValue()
      Get the partitioning column value to use. If a partitionColumnFormula() was set instead of a value that formula is evaluated and captured when effectivePartition() is called.
      Returns:
      the column partition value to use

    • partitionColumnFormula

      @Default @Nullable default String partitionColumnFormula()
      The formula to determine column partition.
      Returns:
      the partitioning column formula.

    • effectivePartition

      default String effectivePartition()
      Compute the effective partition for the merge based upon the partitionColumnFormula() and partitionColumnValue().
      Returns:
      the effective partition

    • threadPoolSize

      @Default default int threadPoolSize()
      Get the number of threads to use for merge. Defaults to the value of the property `MergeParameters.defaultThreadPoolSize`, which defaults to 1
      Returns:
      the number of threads to use

    • maximumConcurrentColumns

      @Default default int maximumConcurrentColumns()
      The maximum number of columns that may be concurrently processed when in maximum throughput mode. In this mode, no matter how many threads are available write jobs will not be enqueued until less than this number of distinct columns are being written. Defaults to unlimited.
      Returns:
      the maximum number of columns that may be written concurrently.

    • lowHeapUsage

      @Default default boolean lowHeapUsage()
      Get if low heap usage mode should be used. Defaults to false.
      Returns:
      true if low heap usage mode is enabled

    • force

      @Default default boolean force()
      Get if merge should overwrite existing data. Defaults to false.
      Returns:
      true if merge should overwrite existing data

    • lateCleanup

      @Default default boolean lateCleanup()
      Get if late cleanup is used. Defaults to false.
      Returns:
      if late cleanup is used

    • allowEmptyInput

      @Default default boolean allowEmptyInput()
      Get if empty input tables are allowed. Defaults to true.
      Returns:
      if empty input tables are allowed

    • sortColumnFormula

      @Nullable @Default default String sortColumnFormula()
      Get the sorting column formula. Defaults to null.
      Returns:
      the sorting column formula

    • storageFormat

      @Nullable @Default default Database.StorageFormat storageFormat()
      Get the Database.StorageFormat to merge into.
      Returns:
      the storage format (or null for system default)

    • codecName

      @Nullable @Default default String codecName()
      Get the compression codec name to use when storageFormat() is Database.StorageFormat.Parquet.
      Returns:
      the compression codec or null for uncompressed/

    • syncMode

      @Default default SyncMode syncMode()
      Get the SyncMode to use.
      Returns:
      the SyncMode

    • sourceTable

      @Nullable Table sourceTable()
      Get the source Table for the merge.
      Returns:
      the source table.

    • check

      @Check @FinalDefault default void check()