Package io.deephaven.enterprise.database.impl

Class DatabaseImpl

java.lang.Object
io.deephaven.enterprise.database.impl.DatabaseImpl
All Implemented Interfaces:
AclSupport, Database, DatabaseInternal, WritableDatabase
public class DatabaseImpl extends Object implements DatabaseInternal, WritableDatabase, AclSupport
A basic implementation of the Database interface.

  • Constructor Details

    • DatabaseImpl

      public DatabaseImpl(@NotNull @NotNull TableDataService tableDataService, @NotNull @NotNull io.deephaven.shadow.enterprise.com.illumon.iris.db.schema.SchemaService schemaService, @NotNull @NotNull GroupProvider groupProvider, @NotNull @NotNull SourceAclProvider sourceAclProvider, @NotNull @NotNull UserContext userContext, @NotNull @NotNull AuditEventLogger auditEventLogger, @NotNull @NotNull ProcessInfo processInfo)

  • Method Details

    • liveTable

      public Table liveTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull TableOptions options)
      Description copied from interface: Database
      Fetch a live Table for the specified namespace and table name.
      Specified by:
      liveTable in interface Database
      Parameters:
      namespace - the Namespace in which the table exists
      tableName - the name of the table in the Namespace.
      options - a TableOptions to set table options.
      Returns:
      a new live Table for the specified parameters.

    • historicalTable

      public Table historicalTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull TableOptions options)
      Description copied from interface: Database
      Fetch a static historical Table from the database.
      Specified by:
      historicalTable in interface Database
      Parameters:
      namespace - the Namespace in which the table exists
      tableName - the name of the table in the Namespace.
      options - a TableOptions to set table options.
      Returns:
      a new static Table for the specified parameters.

    • getTableDefinition

      public TableDefinition getTableDefinition(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: Database
      Fetch a TableDefinition from the database.
      Specified by:
      getTableDefinition in interface Database
      Parameters:
      namespace - the Namespace in which the table exists
      tableName - the name of the table in the Namespace.
      Returns:
      the TableDefinition for the specified parameters.

    • getTableDefinitionTable

      public Table getTableDefinitionTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: Database
      Fetch a table's definition table from the database.
      Specified by:
      getTableDefinitionTable in interface Database
      Parameters:
      namespace - the Namespace in which the table exists
      tableName - the name of the table in the Namespace.
      Returns:
      a Table of the column definitions for the specified parameters.

    • getNamespaces

      public Collection<String> getNamespaces()
      Description copied from interface: Database
      Get the available namespaces.
      Specified by:
      getNamespaces in interface Database
      Returns:
      the list of namespaces

    • getTableNames

      public Collection<String> getTableNames(@Nullable @Nullable String namespace)
      Description copied from interface: Database
      Get the tables within a namespace
      Specified by:
      getTableNames in interface Database
      Parameters:
      namespace - the namespace to interrogate
      Returns:
      the list of tables within namespace

    • getCatalogTable

      public Table getCatalogTable()
      Description copied from interface: Database
      Retrieve a table containing the tables that exist in this database.

      The returned table has a column for "Namespace", "Tablename" and "NamespaceSet".

      Specified by:
      getCatalogTable in interface Database
      Returns:
      a table containing the tables that exist in this database.

    • setCatalog

      public void setCatalog(@NotNull @NotNull io.deephaven.shadow.enterprise.com.illumon.iris.db.tables.databases.DatabaseCatalog databaseCatalog)

    • liveTable

      public Table liveTable(@NotNull @NotNull String setNamespace, @NotNull @NotNull String setTableName, boolean isRefreshing)
      Description copied from interface: AclSupport
      Fetch the specified live (intraday) table.
      Specified by:
      liveTable in interface AclSupport
      Parameters:
      setNamespace - the namespace of the table
      setTableName - the name of the table
      isRefreshing - if the returned table should be refreshing
      Returns:
      the table

    • historicalTable

      public Table historicalTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: Database
      Fetch a static historical Table from the database.
      Specified by:
      historicalTable in interface AclSupport
      Specified by:
      historicalTable in interface Database
      Parameters:
      namespace - the Namespace in which the table exists
      tableName - the name of the table in the Namespace.
      Returns:
      a new static Table for the specified parameters.

    • getUserContext

      @NotNull public @NotNull UserContext getUserContext()
      Specified by:
      getUserContext in interface DatabaseInternal

    • getGroupProvider

      public GroupProvider getGroupProvider()
      Specified by:
      getGroupProvider in interface AclSupport

    • clearLocationCache

      public void clearLocationCache()
      Description copied from interface: Database
      Clear the location cache for all tables. Use this method when some data has been replaced for a table to refresh the location cache and read the replaced data.

      Subsequent calls to Database.historicalTable(String, String), Database.liveTable(String, String) and their variants for all tables will repeat location discovery, which can have significant negative performance consequences.

      Note that you may also disable caching entirely using Database.useLocationCaches(boolean)

      Specified by:
      clearLocationCache in interface Database

    • clearLocationCache

      public void clearLocationCache(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: Database
      Clear the location cache for the specified table. Use this method when some data has been replaced for a table to refresh the location cache and read the replaced data.

      Subsequent calls to Database.historicalTable(String, String), Database.liveTable(String, String) and their variants for this table will repeat location discovery, which can have significant negative performance consequences

      Note that you may also disable caching entirely using Database.useLocationCaches(boolean)

      Specified by:
      clearLocationCache in interface Database
      Parameters:
      namespace - namespace of table to clear cache
      tableName - table name of table to clear cache

    • useLocationCaches

      public void useLocationCaches(boolean useCaching)
      Description copied from interface: Database
      Set whether the database should cache the result of table location discovery and re-use them. This defaults to true
      Specified by:
      useLocationCaches in interface Database
      Parameters:
      useCaching - if location caches should be used.

    • livePartitionedTable

      public PartitionedTable livePartitionedTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull TableOptions options)
      Description copied from interface: Database
      Retrieve the specified live table as a PartitionedTable from the Database. The partitioned table's key columns will be its column and internal partitions.
      Specified by:
      livePartitionedTable in interface Database
      Parameters:
      namespace - the Namespace in which the table exists
      tableName - the name of the table in the Namespace.
      options - a TableOptions to set table options.
      Returns:
      a new PartitionedTable for the specified table

    • historicalPartitionedTable

      public PartitionedTable historicalPartitionedTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull TableOptions options)
      Description copied from interface: Database
      Retrieve the specified historical table as a PartitionedTable from the Database. The result's key columns will be derived from the table's partitioning columns as well as any internal partitions.
      Specified by:
      historicalPartitionedTable in interface Database
      Parameters:
      namespace - the Namespace in which the table exists
      tableName - the name of the table in the Namespace.
      options - a TableOptions to set table options.
      Returns:
      a new PartitionedTable for the specified table

    • addUnpartitionedTable

      public void addUnpartitionedTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull Table table)
      Description copied from interface: WritableDatabase

      Adds an unpartitioned user table.

      Writes an unpartitioned user table to disk. If the namespace does not exist, then it is created. The schema must not already exist.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      addUnpartitionedTable in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      table - table that the definition and data will be based on

    • deleteUnpartitionedTable

      public boolean deleteUnpartitionedTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: WritableDatabase
      Deletes an unpartitioned user table and the schema.

      If the schema does not exist, then data is not deleted. If there is no data but the schema exists, then the schema is deleted.

      The namespace is not removed even if this is the last table in the namespace.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      deleteUnpartitionedTable in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      Returns:
      true if the data was deleted, false if there was no data or preexisting schema

    • addPartitionedTableSchema

      public boolean addPartitionedTableSchema(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull String partitionColumnName, @NotNull @NotNull TableDefinition prototype)
      Description copied from interface: WritableDatabase
      Adds a schema for a partitioned user table.

      The schema is derived from the prototype TableDefinition and the partitionColumnName parameter. The prototype definition must not include a partitioning column.

      If the namespace does not exist, then it is created.

      If the schema already exists and it is identical (this is a stricter check than compatibility; all columns must be present in the same order with the same properties), then the method returns false. If the schema already exists and is not identical, then an error is thrown.

      Specified by:
      addPartitionedTableSchema in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      partitionColumnName - name of the partitioning column
      prototype - table definition to derive schema from
      Returns:
      true if the partitioned table schema was added, false if it already existed

    • updatePartitionedTableSchema

      public boolean updatePartitionedTableSchema(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull TableDefinition prototype)
      Description copied from interface: WritableDatabase
      Updates a preexisting User table schema.

      If the schema does not exist an error is thrown. Not all schema modifications are permitted. The partitioning column may not be changed. Existing columns may not have their type changed. Columns may be added or deleted.

      Note that no data is modified by this operation. Removed columns remain on persistent storage, and added columns are treated as null on read.

      Although each modification in isolation is verified for safety, a sequence of modifications to the schema may be unsafe. For example, deleting a column and adding it back with a new type results in unreadable data.

      Specified by:
      updatePartitionedTableSchema in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      prototype - table definition to derive schema from
      Returns:
      true if the partitioned table schema was updated, false if there was already an identical definition

    • addTablePartition

      public void addTablePartition(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull String partitionColumnValue, @NotNull @NotNull Table table)
      Description copied from interface: WritableDatabase
      Adds a single column partition of data to a partitioned user table.

      The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      addTablePartition in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      partitionColumnValue - value for the partitioning column, e.g. "2015-09-25" for "Date"
      table - table to write data from

    • deleteTablePartition

      public boolean deleteTablePartition(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull String partitionColumnValue)
      Description copied from interface: WritableDatabase
      Deletes a single column partition of data from a partitioned user table.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      deleteTablePartition in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      partitionColumnValue - value for the partitioning column, e.g. "2015-09-25" for "Date"
      Returns:
      true if the partition was deleted, false if there was no partition or preexisting schema

    • appendLiveTableInternal

      public void appendLiveTableInternal(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull String partitionColumnValue, @NotNull @NotNull Table table, @NotNull @NotNull String internalPartitionValue, boolean targetInputTable)

    • appendLiveTable

      public void appendLiveTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull String partitionColumnValue, @NotNull @NotNull Table table)
      Description copied from interface: WritableDatabase
      Appends all rows from a given table to a live user table partition.

      The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column.

      The specified column partition may already exist, or may be created by the system.

      This method is asynchronous. After returning, the data may not be immediately available. It is possible for the write to fail after this method has returned. When multiple workers append to a partition, ordering is imposed outside the worker by other system components.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      appendLiveTable in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      partitionColumnValue - value for the partitioning column, e.g. "2015-09-25" for "Date"
      table - table to append rows from

    • appendLiveTableIncremental

      public SafeCloseable appendLiveTableIncremental(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull String partitionColumnValue, @NotNull @NotNull Table table)
      Description copied from interface: WritableDatabase
      Appends all rows from a given table to a live user table partition. When rows are added to the table, they are additionally appended to the table.

      The input table updates can only have additions and shifts. No modifications or removals are permitted.

      The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column.

      The specified column partition may already exist, or may be created by the system.

      This method is asynchronous, after returning the data may not be immediately available. It is possible for the write to fail after this method has returned. When multiple workers append to a partition, ordering is imposed outside the worker by other system components.

      A reference must be maintained to the returned SafeCloseable to ensure expected functionality; calling SafeCloseable.close() will stop incremental appends, and clean up related resources.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      appendLiveTableIncremental in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      partitionColumnValue - value for the partitioning column, e.g. "2015-09-25" for "Date"
      table - table to append updates from
      Returns:
      the Closeable reference used to ensure and stop expected functionality

    • deleteLiveTablePartition

      public boolean deleteLiveTablePartition(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull String partitionColumnValue)
      Description copied from interface: WritableDatabase
      Delete a partition from a live user table.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      deleteLiveTablePartition in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      partitionColumnValue - value for the partitioning column, e.g. "2015-09-25" for "Date"
      Returns:
      true if the partition was deleted, false if there was no partition or preexisting schema

    • deletePartitionedTable

      public boolean deletePartitionedTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: WritableDatabase
      Delete all partitions, whether direct or live, and the schema, from a partitioned user table.

      All partitions from the table are deleted sequentially. If a partition cannot be deleted, then the operation fails but some data may have already been removed. After all partitions are deleted, then the schema is deleted. If the schema does not exist, then data is not deleted. If there is no data, but the schema exists, then the schema is deleted.

      The namespace is not removed, even if this is the last table in the namespace.

      If a UserTableIOException is thrown, the state of the table is undefined.

      Specified by:
      deletePartitionedTable in interface WritableDatabase
      Parameters:
      namespace - table namespace
      tableName - table name
      Returns:
      true if data was deleted, false if there was no data or preexisting schema

    • addInputTableSchema

      public boolean addInputTableSchema(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull InputTableSpec inputTableSpec)
      Description copied from interface: WritableDatabase
      Add a new input table schema using the InputTableSpec.
      Specified by:
      addInputTableSchema in interface WritableDatabase
      Parameters:
      namespace - the namespace of the input table
      tableName - the name of the input table
      inputTableSpec - the input table specification
      Returns:
      true if the input table schema was added, false if the input table already exists with the same spec

    • addInputTableSchema

      public boolean addInputTableSchema(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, @NotNull @NotNull TableDefinition prototype, @NotNull @NotNull String... keyColNames)
      Description copied from interface: WritableDatabase
      Add a new input table schema using the TableDefinition and specified key column names.
      Specified by:
      addInputTableSchema in interface WritableDatabase
      Parameters:
      namespace - the namespace of the input table
      tableName - the name of the input table
      prototype - TableDefinition to derive the input table spec from
      keyColNames - columns that should be keyed in the input table
      Returns:
      true if the input table schema was added, false if the input table already exists with the same spec

    • updateInputTableSchema

      public boolean updateInputTableSchema(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName, InputTableSpec newInputTableSpec)
      Description copied from interface: WritableDatabase
      Update an existing input table schema using the provided InputTableSpec.

      Retrieve and use a new InputTableUpdater via WritableDatabase.inputTableUpdater(String, String) after updating an input table's specification to ensure proper behavior.

      Specified by:
      updateInputTableSchema in interface WritableDatabase
      Parameters:
      namespace - the namespace of the input table
      tableName - the name of the input table
      newInputTableSpec - the new specification for the input table
      Returns:
      true if the input table schema was updated, false if the input table already exists with the same spec

    • deleteInputTable

      public boolean deleteInputTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: WritableDatabase
      Delete the input table given the namespace and table name.
      Specified by:
      deleteInputTable in interface WritableDatabase
      Parameters:
      namespace - the namespace of the input table
      tableName - the name of the input table
      Returns:
      true if the input table was deleted, false if there was no data or preexisting schema for the input table

    • inputTableUpdater

      public InputTableUpdater inputTableUpdater(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: WritableDatabase
      Retrieve the specified InputTableUpdater, which can be used to programmatically add or delete rows.

      If the caller needs a consistent view between the returned InputTableUpdater and an input table view, the input table view must be derived from InputTableUpdater.table().

      Specified by:
      inputTableUpdater in interface WritableDatabase
      Parameters:
      namespace - the namespace in which the table exists.
      tableName - the name of the table in the namespace.
      Returns:
      an InputTableUpdater for the specified input table.

    • inputTable

      public Table inputTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: WritableDatabase
      Retrieve the specified input table view.

      If the caller needs a consistent view between the returned table and an InputTableUpdater, the InputTableUpdater must be derived from the Table.INPUT_TABLE_ATTRIBUTE from the returned table. Callers may also get a consistent view by calling WritableDatabase.inputTableUpdater(String, String) and deriving the table from InputTableUpdater.table().

      Specified by:
      inputTable in interface WritableDatabase
      Parameters:
      namespace - the namespace in which the table exists.
      tableName - the name of the table in the namespace.
      Returns:
      a table view for the specified input table.

    • inputTableSpecFor

      public InputTableSpec inputTableSpecFor(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: WritableDatabase
      Retrieve the current InputTableSpec for the given namespace and table.
      Specified by:
      inputTableSpecFor in interface WritableDatabase
      Parameters:
      namespace - the namespace of the input table
      tableName - the name of the input table
      Returns:
      the current input table specification for the given namespace and table

    • isInputTable

      public boolean isInputTable(@NotNull @NotNull String namespace, @NotNull @NotNull String tableName)
      Description copied from interface: WritableDatabase
      Determines whether a table is a Core+ input table.
      Specified by:
      isInputTable in interface WritableDatabase
      Parameters:
      namespace - the namespace of the input table
      tableName - the name of the input table
      Returns:
      true if the table is an input table, false if not