deephaven_enterprise.database

This module defines the Database class, which provides an API for the discovery and management of tables in the Deephaven Enterprise data store.

Module Contents

class Database(j_db)[source]

The Database class defines a API for the discovery and management of tables in the Deephaven Enterprise data store.

Parameters:

j_db (jpy.JType)

add_input_table_schema(namespace, table_name, input_table_spec)[source]

Adds a new input table schema using the InputTableSpec.

Parameters:

  • namespace (str) – the namespace of the input table

  • table_name (str) – the name of the input table

  • input_table_spec (InputTableSpec) – the input table specification

Returns:

True if the input table schema was added, False if the input table already exists with the same spec

Raises:

DHError –

Return type:

bool

add_input_table_schema_from_table(namespace, table_name, prototype, key_col_names)[source]

Adds a new input table schema using the Table and specified key column names.

Parameters:

  • namespace (str) – the namespace of the input table

  • table_name (str) – the name of the input table

  • prototype (Table) – the Table to derive the input table spec from

  • key_col_names (list[str]) – columns that should be keyed in the input table

Returns:

True if the input table schema was added, False if the input table already exists with the same spec

Raises:

DHError –

Return type:

bool

add_partitioned_table_schema(namespace, table_name, partition_column_name, prototype, data_indexes=None)[source]

Adds a schema for a partitioned, directly manipulated user table.

The schema for the specified table is derived from the table definition of the prototype table and the partition column name. The prototype table’s definition must not include a partitioning column.

If the namespace does not exist, then it is created.

If the schema already exists, and it is identical (this is a stricter check than compatibility; all columns must be present in the same order with the same properties, and the same data indexes must be defined), then the method returns False. If the schema already exists and is not identical, then an error is thrown.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • partition_column_name (str) – name of the partitioning column

  • prototype (Table) – table with definition to derive schema from

  • data_indexes (Optional[list[list[str]]]) – data indexes to create in the table schema, each expressed as the list of column names comprising the index. Defaults to None.

Returns:

True if the partitioned table schema was added, False if it already existed

Raises:

DHError –

Return type:

bool

add_table_partition(namespace, table_name, partition_column_value, table)[source]

Adds a single partition data of the provided partition column value to a partitioned, directly manipulated user table.

The partition mustn’t already exist. The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column.

Note, if an error occurs during writing, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • partition_column_value (str) – value for the partitioning column, e.g. “2015-09-25” for “Date”

  • table (Table) – table to write data from

Raises:

DHError –

Return type:

None

add_unpartitioned_table(namespace, table_name, table, data_indexes=None)[source]

Adds a unpartitioned, directly manipulated user table in the database.

Writes an unpartitioned user table to disk. If the namespace does not exist, then it is created. Also write out the definition of the table as the schema. The schema must not already exist.

Note, if an error occurs during writing, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • table (Table) – table that the definition and data will be based on

  • data_indexes (Optional[list[list[str]]]) – data indexes to create in the table schema, each expressed as the list of column names comprising the index. Defaults to None.

Raises:

DHError –

Return type:

None

append_live_table(namespace, table_name, partition_column_value, table)[source]

Appends all rows from a given table to a live (centrally managed) user table partition.

The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column.

This method is asynchronous. After returning, the data may not be immediately available. It is possible for the write to fail after this method has returned. When multiple workers append to a partition, ordering is imposed outside the worker by other system components.

Note, if an error is thrown during appending, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • partition_column_value (str) – value for the partitioning column, e.g. “2015-09-25” for “Date”

  • table (Table) – table to append rows from

Raises:

DHError –

Return type:

None

append_live_table_incremental(namespace, table_name, partition_column_value, table)[source]

Appends all rows from the given source table to a live (centrally managed) user table partition. When rows are added to the source table, they are additionally appended to the partition.

The source table updates can only have additions and shifts. No modifications or removals are permitted.

The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column.

This method is asynchronous, after returning the data may not be immediately available. It is possible for the write to fail after this method has returned. When multiple workers append to a partition, ordering is imposed outside the worker by other system components.

A reference must be maintained to the returned AutoClosable object to ensure expected functionality; calling close() on it will stop incremental appends, and clean up related resources.

Note, if an error is thrown during appending, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • partition_column_value (str) – value for the partitioning column, e.g. “2015-09-25” for “Date”

  • table (Table) – table to append updates from

Returns:

the object used to ensure and stop expected functionality

Raises:

DHError –

Return type:

deephaven.jcompat.AutoCloseable

catalog_table()[source]

Returns a table of the available tables. The result table contains 3 string type columns: Namespace, TableName, and NamespaceSet which is either System or User.

Returns:

a table

Return type:

deephaven.table.Table

delete_input_table(namespace, table_name)[source]

Deletes the input table given the namespace and table name.

Parameters:

  • namespace (str) – the namespace of the input table

  • table_name (str) – the name of the input table

Returns:

True if the input table was deleted, False if there was no data or preexisting schema for the input table

Raises:

DHError –

Return type:

bool

delete_live_table_partition(namespace, table_name, partition_column_value)[source]

Deletes a partition from a live (centrally managed) user table.

Note, if an error is thrown during deleting, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • partition_column_value (str) – value for the partitioning column, e.g. “2015-09-25” for “Date”

Returns:

True if the partition was deleted, False if there was no partition or preexisting schema

Raises:

DHError –

Return type:

bool

delete_partitioned_table(namespace, table_name)[source]

Deletes all partitions, whether direct manipulated or live (centrally managed), and the schema, from a partitioned user table.

All partitions from the table are deleted sequentially. If a partition cannot be deleted, then the operation fails but some data may have already been removed. After all partitions are deleted, then the schema is deleted. If the schema does not exist, then data is not deleted. If there is no data, but the schema exists, then the schema is deleted.

The namespace is not removed, even if this is the last table in the namespace.

Note, if an error is thrown during deleting, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

Returns:

True if data was deleted, False if there was no data or preexisting schema

Raises:

DHError –

Return type:

bool

delete_table_partition(namespace, table_name, partition_column_value)[source]

Deletes the partition of the provided partition column value from a partitioned, directly manipulated user table.

Note, if an error is thrown while deleting, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • partition_column_value (str) – value for the partitioning column, e.g. “2015-09-25” for “Date”

Returns:

True if the partition was deleted, False if there was no partition or preexisting schema

Return type:

bool

delete_unpartitioned_table(namespace, table_name)[source]

Deletes a unpartitioned, directly manipulated user table and its associated schema.

If the schema does not exist, then data is not deleted. If there is no data but the schema exists, then the schema is deleted.

The namespace is not removed even if this is the last table in the namespace.

Note, if an error occurs during deleting, the state of the table is undefined.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

Returns:

True if the data was deleted, False if there was no data or preexisting schema

Raises:

DHError –

Return type:

bool

has_table(namespace, table_name)[source]

Identifies if the table is defined in the Database. and visible to the user.

Parameters:

  • namespace (str) – the namespace

  • table_name (str) – the table

Returns:

True if the schema exists and is visible to the user, else False

Return type:

bool

historical_partitioned_table(namespace, table_name)[source]

Retrieves the specified historical table as a partitioned table from the Database. The result’s key columns will be derived from the table’s partitioning columns as well as any internal partitions.

Parameters:

  • namespace (str) – the namespace

  • table_name (str) – the table name

Returns:

a PartitionedTable object

Return type:

deephaven.table.PartitionedTable

historical_table(namespace, table_name, internal_partition_column=None)[source]

Retrieves a historical Table for the specified namespace and table name.

Parameters:

  • namespace (str) – the namespace

  • table_name (str) – the table name

  • internal_partition_column (str) – Set the name of the internal partition column, defaults to None, meaning no internal partition column is included

Returns:

a Table object

Return type:

deephaven.table.Table

input_table(namespace, table_name)[source]

Retrieves the specified input table view.

Parameters:

  • namespace (str) – the namespace in which the table exists

  • table_name (str) – the name of the table in the namespace

Returns:

a InputTable object

Raises:

DHError –

Return type:

deephaven.table_factory.InputTable

input_table_spec_for(namespace, table_name)[source]

Retrieves the current InputTableSpec for the given namespace and table.

Parameters:

  • namespace (str) – the namespace of the input table

  • table_name (str) – the name of the input table

Returns:

a InputTableSpec object

Raises:

DHError –

Return type:

deephaven_enterprise.input_tables.InputTableSpec

live_partitioned_table(namespace, table_name)[source]

Retrieves the specified live table as a partitioned table from the Database. The result’s key columns will be derived from the table’s partitioning columns as well as any internal partitions.

Parameters:

  • namespace (str) – the namespace

  • table_name (str) – the table name

Returns:

a PartitionedTable object

Return type:

deephaven.table.PartitionedTable

live_table(namespace, table_name, is_refreshing=None, is_blink=False, internal_partition_column=None)[source]

Retrieves a live Table for the specified namespace and table name.

Parameters:

  • namespace (str) – the namespace

  • table_name (str) – the table name

  • is_refreshing (bool) – True if the returned table should be refreshing, defaults to None, meaning to use the value of JVM property RunAndDoneSetupQuery.liveTables which defaults to False

  • is_blink (bool) – True if the returned table should be a blink table, defaults to False

  • internal_partition_column (str) – Set the name of the internal partition column, defaults to None, meaning no internal partition column is included

Returns:

a Table object

Return type:

deephaven.table.Table

namespaces()[source]

Returns the list of namespaces in this database.

Returns:

the list of namespaces

Return type:

list[str]

table_definition(namespace, table_name)[source]

Returns a Table containing the column definitions for the table with the specified namespace and table name.

Parameters:

  • namespace (str) – the namespace

  • table_name (str) – the table name

Returns:

a Table object

Return type:

deephaven.table.Table

table_names(namespace)[source]

Returns the list tables within a namespace.

Parameters:

namespace (str) – the namespace

Returns:

the list of table names within namespace

Return type:

list[str]

update_input_table_schema(namespace, table_name, input_table_spec)[source]

Updates an existing input table schema using the provided InputTableSpec.

Retrieve and use a new InputTable via input_table after updating an input table’s specification to ensure proper behavior.

When the requested update is invalid, e.g. an input table already exists but with a different spec, an exception will be raised.

Parameters:

  • namespace (str) – the namespace of the input table

  • table_name (str) – the name of the input table

  • input_table_spec (InputTableSpec) – the new specification for the input table

Returns:

True if the input table schema was updated, False if the input table already exists with the same spec

Raises:

DHError –

Return type:

bool

update_partitioned_table_schema(namespace, table_name, prototype, data_indexes=None)[source]

Updates the schema of a preexisting partitioned, directly manipulated user table.

If the schema does not exist an error is thrown. Not all schema modifications are permitted. The partitioning column may not be changed. Existing columns may not have their type changed. Columns may be added or deleted.

Note that no data is modified by this operation. Removed columns remain on persistent storage, and added columns are treated as null on read.

Note also that although each modification in isolation is verified for safety, a sequence of modifications to the schema may be unsafe. For example, deleting a column and adding it back with a new type results in unreadable data.

Parameters:

  • namespace (str) – table namespace

  • table_name (str) – table name

  • prototype (Table) – table with definition to derive schema from

  • data_indexes (Optional[list[list[str]]]) – data indexes to create in the table schema, each expressed as the list of column names comprising the index. Defaults to None.

Returns:

True if the partitioned table definition was updated, False if there was already an identical definition

Raises:

DHError –

Return type:

bool