Package io.deephaven.engine.table

Interface PartitionedTable

All Superinterfaces:
LivenessManager, LivenessNode, LivenessReferent, LogOutputAppendable
All Known Implementing Classes:
PartitionedTableImpl, SourcePartitionedTable
public interface PartitionedTable extends LivenessNode, LogOutputAppendable

Interface for working with partitioned tables.

A partitioned table is a Table with one or more columns containing non-null, like-defined constituent tables, optionally with "key" columns defined to allow partitionedTransform(PartitionedTable, BinaryOperator, Dependency...) or proxied joins with other like-keyed partitioned tables.

Note that partitioned tables should depend on and manage their refreshing constituents.

  • Method Details

    • table

      @ConcurrentMethod Table table()
      Get the "raw" partitioned table underlying this PartitionedTable.

      The raw table can be converted back into a partitioned table using PartitionedTableFactory.of(Table) or PartitionedTableFactory.of(Table, Collection, boolean, String, TableDefinition, boolean).

      Returns:
      The underlying partitioned table

    • keyColumnNames

      @ConcurrentMethod Set<String> keyColumnNames()
      Get the names of all "key" columns that are part of table().getDefinition(). If there are no key columns, the result will be empty. This set is explicitly ordered.
      Returns:
      The key column names

    • uniqueKeys

      @ConcurrentMethod boolean uniqueKeys()

      Are the keys (key column values for a row considered as a tuple) in the underlying partitioned table unique?

      If keys are unique, one can expect that table().selectDistinct(keyColumnNames.toArray(String[]::new)) is equivalent to table().view(keyColumnNames.toArray(String[]::new)).

      Returns:
      Whether the keys in the underlying partitioned table are unique

    • constituentColumnName

      @ConcurrentMethod String constituentColumnName()
      Get the name of the "constituent" column of tables.
      Returns:
      The constituent column name

    • constituentDefinition

      @ConcurrentMethod TableDefinition constituentDefinition()
      Get a definition shared by or mutually compatible with all constituent tables.
      Returns:
      The constituent definition

    • constituentChangesPermitted

      @ConcurrentMethod boolean constituentChangesPermitted()

      Can the constituents of the underlying partitioned table change?

      This is completely unrelated to whether the constituents themselves are refreshing, or whether the underlying partitioned table is refreshing. Note that the underlying partitioned table must be refreshing if it contains any refreshing constituents.

      PartitionedTables that specify constituentChangesPermitted() == false must be guaranteed to never change their constituents. Formally, it is expected that table() will never report additions, removals, or shifts, and that any modifications reported will not change values in the constituent column (that is, table().getColumnSource(constituentColumnName())).

      Returns:
      Whether the constituents of the underlying partitioned table can change

    • proxy

      @FinalDefault @ConcurrentMethod default PartitionedTable.Proxy proxy()
      Same as proxy(true, true).
      Returns:
      A proxy that allows table operations to be applied to the constituent tables of this PartitionedTable
      See Also:

    • proxy

      @ConcurrentMethod PartitionedTable.Proxy proxy(boolean requireMatchingKeys, boolean sanityCheckJoinOperations)

      Make a proxy that allows table operations to be applied to the constituent tables of this PartitionedTable.

      Each operation thus applied will produce a new PartitionedTable with the results as in transform(UnaryOperator, Dependency...) or partitionedTransform(PartitionedTable, BinaryOperator, Dependency...), and return a new proxy to that PartitionedTable.

      Parameters:
      requireMatchingKeys - Whether to ensure that both partitioned tables have all the same keys present when a proxied operation uses this and another PartitionedTable as inputs for a partitioned transform
      sanityCheckJoinOperations - Whether to check that proxied join operations will only find a given join key in one constituent table for this and the table argument if it is also a proxy
      Returns:
      A proxy that allows table operations to be applied to the constituent tables of this PartitionedTable

    • merge

      Table merge()
      Make a new Table that contains the rows from all the constituent tables of this PartitionedTable, in the same relative order as the underlying partitioned table and its constituents. If constituent tables contain extra columns not in the constituent definition, those columns will be ignored. If constituent tables are missing columns in the constituent definition, the corresponding output rows will be null.
      Returns:
      A merged representation of the constituent tables

    • filter

      @ConcurrentMethod PartitionedTable filter(Collection<? extends Filter> filters)

      Make a new PartitionedTable from the result of applying filters to the underlying partitioned table.

      filters must not reference the constituent column.

      Parameters:
      filters - The filters to apply. Must not reference the constituent column.
      Returns:
      The filtered PartitionedTable

    • sort

      @ConcurrentMethod PartitionedTable sort(Collection<SortColumn> sortColumns)

      Make a new PartitionedTable from the result of applying sortColumns to the underlying partitioned table.

      sortColumns must not reference the constituent column.

      Parameters:
      sortColumns - The columns to sort by. Must not reference the constituent column.
      Returns:
      The sorted PartitionedTable

    • transform

      default PartitionedTable transform(@NotNull @NotNull UnaryOperator<Table> transformer, @NotNull @NotNull NotificationQueue.Dependency... dependencies)

      Apply transformer to all constituent tables, and produce a new PartitionedTable containing the results.

      This overload uses the enclosing ExecutionContext and expects transformer to produce refreshing results if and only if this PartitionedTable's underlying table is refreshing.

      Parameters:
      transformer - The UnaryOperator to apply to all constituent tables
      dependencies - Additional dependencies that must be satisfied before applying transformer to added or modified constituents during update processing; use this when transformer uses additional Table or PartitionedTable inputs besides the constituents of this
      Returns:
      The new PartitionedTable containing the resulting constituents
      Throws:
      IllegalStateException - On instantiation or update if !table().isRefreshing() and transformer produces a refreshing result for any constituent
      ApiNote:
      transformer must be stateless, safe for concurrent use, and able to return a valid result for an empty input table. It is required to install an ExecutionContext to access any QueryLibrary/QueryScope/QueryCompiler functionality from the transformer.

    • transform

      PartitionedTable transform(@Nullable @Nullable ExecutionContext executionContext, @NotNull @NotNull UnaryOperator<Table> transformer, boolean expectRefreshingResults, @NotNull @NotNull NotificationQueue.Dependency... dependencies)

      Apply transformer to all constituent tables, and produce a new PartitionedTable containing the results. The transformer will be invoked in the provided ExecutionContext.

      transformer must be stateless, safe for concurrent use, and able to return a valid result for an empty input table.

      Parameters:
      executionContext - The ExecutionContext to use for the transformer
      transformer - The UnaryOperator to apply to all constituent tables
      expectRefreshingResults - Whether to expect that the results of applying transformer may be refreshing. If true, the resulting PartitionedTable will always be backed by a refreshing table. This hint is important for transforms to static inputs that might produce refreshing output, in order to ensure correct liveness management; incorrectly specifying false will result in exceptions.
      dependencies - Additional dependencies that must be satisfied before applying transformer to added or modified constituents during update processing; use this when transformer uses additional Table or PartitionedTable inputs besides the constituents of this
      Returns:
      The new PartitionedTable containing the resulting constituents
      Throws:
      IllegalStateException - On instantiation or update if !table().isRefreshing() && !expectRefreshingResults and transformer produces a refreshing result for any constituent

    • partitionedTransform

      default PartitionedTable partitionedTransform(@NotNull @NotNull PartitionedTable other, @NotNull @NotNull BinaryOperator<Table> transformer, @NotNull @NotNull NotificationQueue.Dependency... dependencies)

      Apply transformer to all constituent tables found in this and other with the same key column values, and produce a new PartitionedTable containing the results.

      Note that other's key columns must match this PartitionedTable's key columns. Two matching mechanisms are supported, and will be attempted in the order listed:

      1. Match by column name. Both PartitionedTables must have all the same key column names. Like-named columns must have the same data type and component type.
      2. Match by column order. Both PartitionedTables must have their matchable columns in the same order within their key column names. Like-positioned columns must have the same data type and component type.

      This overload uses the enclosing ExecutionContext and expects transformer to produce refreshing results if and only if this or other has a refreshing underlying table.

      Parameters:
      other - The other PartitionedTable to find constituents in
      transformer - The BinaryOperator to apply to all pairs of constituent tables
      dependencies - Additional dependencies that must be satisfied before applying transformer to added, modified, or newly-matched constituents during update processing; use this when transformer uses additional Table or PartitionedTable inputs besides the constituents of this or other
      Returns:
      The new PartitionedTable containing the resulting constituents
      Throws:
      IllegalStateException - On instantiation or update if !table().isRefreshing() && !other.table().isRefreshing() and transformer produces a refreshing result for any constituent
      ApiNote:
      transformer must be stateless, safe for concurrent use, and able to return a valid result for empty input tables. It is required to install an ExecutionContext to access any QueryLibrary/QueryScope/QueryCompiler functionality from the transformer.

    • partitionedTransform

      PartitionedTable partitionedTransform(@NotNull @NotNull PartitionedTable other, @Nullable @Nullable ExecutionContext executionContext, @NotNull @NotNull BinaryOperator<Table> transformer, boolean expectRefreshingResults, @NotNull @NotNull NotificationQueue.Dependency... dependencies)

      Apply transformer to all constituent tables found in this and other with the same key column values, and produce a new PartitionedTable containing the results. The transformer will be invoked in the provided ExecutionContext.

      Note that other's key columns must match this PartitionedTable's key columns. Two matching mechanisms are supported, and will be attempted in the order listed:

      1. Match by column name. Both PartitionedTables must have all the same key column names. Like-named columns must have the same data type and component type.
      2. Match by column order. Both PartitionedTables must have their matchable columns in the same order within their key column names. Like-positioned columns must have the same data type and component type.

      transformer must be stateless, safe for concurrent use, and able to return a valid result for empty input tables. It is required to install an ExecutionContext to access any QueryLibrary/QueryScope/QueryCompiler functionality from the transformer.

      Parameters:
      other - The other PartitionedTable to find constituents in
      executionContext - The ExecutionContext to use for the transformer
      transformer - The BinaryOperator to apply to all pairs of constituent tables
      expectRefreshingResults - Whether to expect that the results of applying transformer may be refreshing. If true, the resulting PartitionedTable will always be backed by a refreshing table. This hint is important for transforms to static inputs that might produce refreshing output, in order to ensure correct liveness management; incorrectly specifying false will result in exceptions.
      dependencies - Additional dependencies that must be satisfied before applying transformer to added, modified, or newly-matched constituents during update processing; use this when transformer uses additional Table or PartitionedTable inputs besides the constituents of this or other
      Returns:
      The new PartitionedTable containing the resulting constituents
      Throws:
      IllegalStateException - On instantiation or update if !table().isRefreshing() && !other.table().isRefreshing() && !expectRefreshingResults and transformer produces a refreshing result for any constituent

    • constituentFor

      @ConcurrentMethod Table constituentFor(@NotNull @NotNull Object... keyColumnValues)

      Get a single constituent by its corresponding key column values.

      The keyColumnValues can be thought of as a tuple constraining the values for the corresponding key columns for the result row. If there are no matching rows, the result is null. If there are multiple matching rows, an UnsupportedOperationException is thrown.

      The result will be managed by the enclosing liveness scope.

      Note that if constituentChangesPermitted(), this method may return different results if invoked multiple times.

      Parameters:
      keyColumnValues - Ordered, boxed values for the key columns in the same order as keyColumnNames()
      Returns:
      The constituent at the single row in table() matching the keyColumnValues, or null if no matches were found
      Throws:
      IllegalArgumentException - If keyColumnValues.length != keyColumnNames().size()
      UnsupportedOperationException - If multiple matching rows for the keyColumnValues were found

    • constituents

      @ConcurrentMethod Table[] constituents()
      Get all the current constituents.

      The results will be managed by the enclosing liveness scope.

      Note that if constituentChangesPermitted(), this method may return different results if invoked multiple times.

      Returns:
      An array of all current constituents