Package io.deephaven.enterprise.derivedtablewriter

Class DerivedTableWriter.Options

java.lang.Object
io.deephaven.enterprise.derivedtablewriter.DerivedTableWriter.Options
Enclosing class:
DerivedTableWriter
public static class DerivedTableWriter.Options extends Object
Represents Options for ingesting a table to a DIS partition.

  • Constructor Details

    • Options

      public Options()

  • Method Details

    • sequenceColumn

      public DerivedTableWriter.Options sequenceColumn(String sequenceColumn)
      Specify a column in the source table that contains a sequence number.

      If ingestion is interrupted, on resume the last commited sequence number is read from the Deephaven checkpoint record, and records less than or equal to that sequence number are discarded. The sequence number must be ascending, ingestion fails if the the sequence number does not increase on each row. The sequence number need not be sequential, gaps are ignored. This allows you to filter a table before logging it.

      If you do not specify a sequence number, then all new rows presented to the ingester are ingested on startup. If you are deriving data, for example from an add-only table, then without a sequence number data is duplicated on restart.

      Parameters:
      sequenceColumn - the column name of the sequence number
      Returns:
      this Options structure

    • transactionsEnabled

      public DerivedTableWriter.Options transactionsEnabled(boolean transactionsEnabled)
      Should transactions be enabled for each set of data written? (defaults to false)

      A Deephaven transaction ensures that a set of rows are written and presented to downstream queries atomically. If you have data that has meaning across rows (e.g., snapshots of positions), then you should enable transactions on the derived table. This is only meaningful if your source data is also presented atomically.

      Parameters:
      transactionsEnabled - true to enable transactions, false to disable transactions
      Returns:
      this Options structure

    • logInitial

      public DerivedTableWriter.Options logInitial(boolean logInitial)
      Should the initial image of the table be logged?
      Parameters:
      logInitial - true to log the initial rows in the table, false to only log new rows
      Returns:
      this Options structure

    • chunkSize

      public DerivedTableWriter.Options chunkSize(int chunkSize)
      How many rows should be read at one time? (defaults to 4096)
      Parameters:
      chunkSize - the size of chunks (in rows) to be read from the source table and written to the destination table
      Returns:
      this Options structure

    • lastBy

      @ScriptApi public DerivedTableWriter.Options lastBy(String name, Collection<String> keyColumnNames)
      Specify that this DIS should produce a lastBy view of the table with the given key columns using a CoreLastByTableImportState under the hood.
      Parameters:
      name - the name of the lastBy view
      keyColumnNames - the key columns for the lastBy view
      Returns:
      this builder

    • lastBy

      @ScriptApi public DerivedTableWriter.Options lastBy(Collection<String> keyColumnNames)
      Specify that this DIS should produce a lastBy view of the table with the given key columns using a CoreLastByTableImportState under the hood.
      Parameters:
      keyColumnNames - the key columns for the lastBy view
      Returns:
      this builder