Package io.deephaven.engine.table.impl.remote

Class ConstructSnapshot

java.lang.Object
io.deephaven.engine.table.impl.remote.ConstructSnapshot
public class ConstructSnapshot extends Object
A Set of static utilities for computing values from a table while avoiding the use of an update graph lock. This class supports snapshots in both position space and key space.

  • Field Details

    • SNAPSHOT_CHUNK_SIZE

      public static final int SNAPSHOT_CHUNK_SIZE

  • Constructor Details

    • ConstructSnapshot

      public ConstructSnapshot()

  • Method Details

    • state

      public static ConstructSnapshot.State state()
      Get the currently-active snapshot state.
      Returns:
      the currently-active snapshot state

    • concurrentAttemptInconsistent

      public static boolean concurrentAttemptInconsistent()
      Test that determines whether the currently-active concurrent snapshot attempt has become inconsistent. Always returns false if there is no snapshot attempt active, or if there is a locked attempt active (necessarily at lower depth than the lowest concurrent attempt).

      Equivalent to state().concurrentAttemptInconsistent().

      Returns:
      Whether the clock or sources have changed in such a way as to make the currently-active concurrent snapshot attempt inconsistent
      See Also:

    • failIfConcurrentAttemptInconsistent

      public static void failIfConcurrentAttemptInconsistent()
      Check that fails if the currently-active concurrent snapshot attempt has become inconsistent. source. This is a no-op if there is no snapshot attempt active, or if there is a locked attempt active (necessarily at lower depth than the lowest concurrent attempt).

      Equivalent to state().failIfConcurrentAttemptInconsistent().

      Throws:
      ConstructSnapshot.SnapshotInconsistentException - If the currently-active concurrent snapshot attempt has become inconsistent
      See Also:

    • maybeWaitForSatisfaction

      public static void maybeWaitForSatisfaction(@Nullable NotificationQueue.Dependency dependency)
      Wait for a dependency to become satisfied on the current cycle if we're trying to use current values for the currently-active concurrent snapshot attempt. This is a no-op if there is no snapshot attempt active, or if there is a locked attempt active (necessarily at lower depth than the lowest concurrent attempt).

      Equivalent to state().maybeWaitForSatisfaction(dependency).

      Parameters:
      dependency - The dependency, which may be null in order to avoid redundant checks in calling code
      Throws:
      ConstructSnapshot.SnapshotInconsistentException - If we cannot wait for this dependency on the current step because the step changed
      See Also:

    • getConcurrentAttemptClockValue

      public static long getConcurrentAttemptClockValue()
      Return the currently-active concurrent snapshot attempt's "before" clock value, or zero if there is no concurrent attempt active.

      Equivalent to state().getConcurrentAttemptClockValue().

      Returns:
      The concurrent snapshot attempt's "before" clock value, or zero
      See Also:

    • appendConcurrentAttemptClockInfo

      public static LogOutput appendConcurrentAttemptClockInfo(@NotNull @NotNull LogOutput logOutput)
      Append clock info that pertains to the concurrent attempt state to logOutput.

      Equivalent to state().appendConcurrentAttemptClockInfo(logOutput).

      Parameters:
      logOutput - The LogOutput
      Returns:
      logOutput
      See Also:

    • constructInitialSnapshot

      public static InitialSnapshot constructInitialSnapshot(@NotNull @NotNull Object logIdentityObject, @NotNull @NotNull BaseTable<?> table)
      Create a snapshot of the entire table specified. Note that this method is notification-oblivious, i.e. it makes no attempt to ensure that notifications are not missed.
      Parameters:
      logIdentityObject - An object used to prepend to log rows.
      table - the table to snapshot.
      Returns:
      a snapshot of the entire base table.

    • constructInitialSnapshot

      public static InitialSnapshot constructInitialSnapshot(@NotNull @NotNull Object logIdentityObject, @NotNull @NotNull BaseTable<?> table, @Nullable @Nullable BitSet columnsToSerialize, @Nullable @Nullable RowSet keysToSnapshot)
      Create a snapshot of the specified table using a set of requested columns and keys. Note that this method uses a RowSet that is in key space, and that it is notification-oblivious, i.e. it makes no attempt to ensure that notifications are not missed.
      Parameters:
      logIdentityObject - An object used to prepend to log rows.
      table - the table to snapshot.
      columnsToSerialize - A BitSet of columns to include, null for all
      keysToSnapshot - An RowSet of keys within the table to include, null for all
      Returns:
      a snapshot of the entire base table.

    • constructInitialSnapshotInPositionSpace

      public static InitialSnapshot constructInitialSnapshotInPositionSpace(@NotNull @NotNull Object logIdentityObject, @NotNull @NotNull BaseTable<?> table, @Nullable @Nullable BitSet columnsToSerialize, @Nullable @Nullable RowSet positionsToSnapshot)
      Create a snapshot of the specified table using a set of requested columns and positions. Note that this method uses a RowSet that is in position space, and that it is notification-oblivious, i.e. it makes no attempt to ensure that notifications are not missed.
      Parameters:
      logIdentityObject - An object used to prepend to log rows.
      table - the table to snapshot.
      columnsToSerialize - A BitSet of columns to include, null for all
      positionsToSnapshot - An RowSet of positions within the table to include, null for all
      Returns:
      a snapshot of the entire base table.

    • constructBackplaneSnapshot

      public static BarrageMessage constructBackplaneSnapshot(@NotNull @NotNull Object logIdentityObject, @NotNull @NotNull BaseTable<?> table)
      Create a snapshot of the specified table including all columns and rows. Note that this method is notification-oblivious, i.e. it makes no attempt to ensure that notifications are not missed.
      Parameters:
      logIdentityObject - An object used to prepend to log rows.
      table - the table to snapshot.
      Returns:
      a snapshot of the entire base table.

    • constructBackplaneSnapshotInPositionSpace

      public static BarrageMessage constructBackplaneSnapshotInPositionSpace(@NotNull @NotNull Object logIdentityObject, @NotNull @NotNull BaseTable<?> table, @Nullable @Nullable BitSet columnsToSerialize, @Nullable @Nullable RowSequence positionsToSnapshot, @Nullable @Nullable RowSequence reversePositionsToSnapshot)
      Create a snapshot of the specified table using a set of requested columns and positions. Note that this method uses a RowSet that is in position space, and that it is notification-oblivious, i.e. it makes no attempt to ensure that notifications are not missed.
      Parameters:
      logIdentityObject - An object used to prepend to log rows.
      table - the table to snapshot.
      columnsToSerialize - A BitSet of columns to include, null for all
      positionsToSnapshot - An RowSet of positions within the table to include, null for all
      Returns:
      a snapshot of the entire base table.

    • constructBackplaneSnapshotInPositionSpace

      public static BarrageMessage constructBackplaneSnapshotInPositionSpace(@NotNull @NotNull Object logIdentityObject, @NotNull @NotNull BaseTable<?> table, @Nullable @Nullable BitSet columnsToSerialize, @Nullable @Nullable RowSequence positionsToSnapshot, @Nullable @Nullable RowSequence reversePositionsToSnapshot, @NotNull @NotNull ConstructSnapshot.SnapshotControl control)
      Create a snapshot of the specified table using a set of requested columns and positions. Note that this method uses a RowSet that is in position space.
      Parameters:
      logIdentityObject - An object used to prepend to log rows.
      table - the table to snapshot.
      columnsToSerialize - A BitSet of columns to include, null for all
      positionsToSnapshot - A RowSequence of positions within the table to include, null for all
      reversePositionsToSnapshot - A RowSequence of reverse positions within the table to include, null for all
      control - A ConstructSnapshot.SnapshotControl to define the parameters and consistency for this snapshot
      Returns:
      a snapshot of the entire base table.

    • constructInitialSnapshots

      public static List<InitialSnapshot> constructInitialSnapshots(@NotNull @NotNull Object logIdentityObject, @NotNull @NotNull BaseTable<?>... tables)
      Constructs InitialSnapshots for the entirety of the tables. Note that this method is notification-oblivious, i.e. it makes no attempt to ensure that notifications are not missed.
      Parameters:
      logIdentityObject - identifier prefixing the log message
      tables - tables to snapshot
      Returns:
      list of the resulting InitialSnapshots

    • makeSnapshotControl

      public static ConstructSnapshot.SnapshotControl makeSnapshotControl(@NotNull @NotNull UpdateGraph updateGraph, @NotNull @NotNull ConstructSnapshot.UsePreviousValues usePreviousValues, @NotNull @NotNull ConstructSnapshot.SnapshotConsistent snapshotConsistent, @Nullable @Nullable ConstructSnapshot.SnapshotCompletedConsistently snapshotCompletedConsistently)
      Make a ConstructSnapshot.SnapshotControl from individual function objects.
      Parameters:
      updateGraph - The UpdateGraph for the snapshot
      usePreviousValues - The ConstructSnapshot.UsePreviousValues to use
      snapshotConsistent - The ConstructSnapshot.SnapshotConsistent to use
      snapshotCompletedConsistently - The ConstructSnapshot.SnapshotCompletedConsistently to use, or null to use snapshotConsistent

    • makeSnapshotControl

      public static ConstructSnapshot.SnapshotControl makeSnapshotControl(boolean notificationAware, boolean refreshing, @NotNull @NotNull NotificationStepSource source)
      Make a default ConstructSnapshot.SnapshotControl for a single source.
      Parameters:
      notificationAware - Whether the result should be concerned with not missing notifications
      refreshing - Whether the data source (usually a Table table) is refreshing (vs static)
      source - The source
      Returns:
      An appropriate ConstructSnapshot.SnapshotControl

    • makeSnapshotControl

      public static ConstructSnapshot.SnapshotControl makeSnapshotControl(boolean notificationAware, boolean refreshing, @NotNull @NotNull NotificationStepSource... sources)
      Make a default ConstructSnapshot.SnapshotControl for one or more sources.
      Parameters:
      notificationAware - Whether the result should be concerned with not missing notifications
      refreshing - Whether any of the data sources (usually tables) are refreshing (vs static)
      sources - The sources
      Returns:
      An appropriate ConstructSnapshot.SnapshotControl

    • callDataSnapshotFunction

      public static long callDataSnapshotFunction(@NotNull @NotNull String logPrefix, @NotNull @NotNull ConstructSnapshot.SnapshotControl control, @NotNull @NotNull ConstructSnapshot.SnapshotFunction function)
      Invokes the snapshot function in a loop until it succeeds with provably consistent results, or until MAX_CONCURRENT_ATTEMPTS or MAX_CONCURRENT_ATTEMPT_DURATION_MILLIS are exceeded. Falls back to acquiring a shared update graph lock for a final attempt.
      Parameters:
      logPrefix - A prefix for our log messages
      control - A ConstructSnapshot.SnapshotControl to define the parameters and consistency for this snapshot
      function - The function to execute
      Returns:
      The logical clock step that applied to this snapshot

    • callDataSnapshotFunction

      public static long callDataSnapshotFunction(@NotNull @NotNull LogOutputAppendable logPrefix, @NotNull @NotNull ConstructSnapshot.SnapshotControl control, @NotNull @NotNull ConstructSnapshot.SnapshotFunction function)
      Invokes the snapshot function in a loop until it succeeds with provably consistent results, or until MAX_CONCURRENT_ATTEMPTS or MAX_CONCURRENT_ATTEMPT_DURATION_MILLIS are exceeded. Falls back to acquiring a shared update graph lock for a final attempt.

      The supplied ConstructSnapshot.SnapshotControl's usePreviousValues will be invoked at the start of any snapshot attempt, and its snapshotCompletedConsistently will be invoked at the end of any snapshot attempt that is not provably inconsistent.

      If the supplied ConstructSnapshot.SnapshotControl provides a null UpdateGraph, then this method will perform a static snapshot without locks or retrying. In this case, the ConstructSnapshot.SnapshotControl's usePreviousValues must return false, snapshotCompletedConsistently must return true, and the NULL_CLOCK_VALUE will be supplied to usePreviousValues and snapshotCompletedConsistently.

      Parameters:
      logPrefix - A prefix for our log messages
      control - A ConstructSnapshot.SnapshotControl to define the parameters and consistency for this snapshot
      function - The function to execute
      Returns:
      The logical clock step that applied to this snapshot; -1L for static snapshots

    • estimateSnapshotSize

      public static long estimateSnapshotSize(@NotNull @NotNull Table table)
      Estimate the size of a complete table snapshot in bytes.
      Parameters:
      table - the table to estimate
      Returns:
      the estimated snapshot size in bytes.

    • estimateSnapshotSize

      public static long estimateSnapshotSize(@NotNull @NotNull TableDefinition tableDefinition, @NotNull @NotNull BitSet columns, long rowCount)
      Make a rough guess at the size of a snapshot, using the column types and common column names. The use case is when a user requests something from the GUI; we'd like to know if it is ridiculous before actually doing it.
      Parameters:
      tableDefinition - the table definition
      columns - a bitset indicating which columns are included
      rowCount - how many rows of this data we'll be snapshotting
      Returns:
      the estimated size of the snapshot