deephaven_enterprise.database ============================= .. py:module:: deephaven_enterprise.database .. autoapi-nested-parse:: This module defines the Database class, which provides an API for the discovery and management of tables in the Deephaven Enterprise data store. Module Contents --------------- .. py:class:: Database(j_db) The Database class defines a API for the discovery and management of tables in the Deephaven Enterprise data store. .. py:method:: add_input_table_schema(namespace, table_name, input_table_spec) Adds a new input table schema using the InputTableSpec. :param namespace: the namespace of the input table :type namespace: str :param table_name: the name of the input table :type table_name: str :param input_table_spec: the input table specification :type input_table_spec: InputTableSpec :returns: True if the input table schema was added, False if the input table already exists with the same spec :raises DHError: .. py:method:: add_input_table_schema_from_table(namespace, table_name, prototype, key_col_names) Adds a new input table schema using the Table and specified key column names. :param namespace: the namespace of the input table :type namespace: str :param table_name: the name of the input table :type table_name: str :param prototype: the Table to derive the input table spec from :type prototype: Table :param key_col_names: columns that should be keyed in the input table :type key_col_names: list[str] :returns: True if the input table schema was added, False if the input table already exists with the same spec :raises DHError: .. py:method:: add_partitioned_table_schema(namespace, table_name, partition_column_name, prototype, data_indexes = None) Adds a schema for a partitioned, directly manipulated user table. The schema for the specified table is derived from the table definition of the prototype table and the partition column name. The prototype table’s definition must not include a partitioning column. If the namespace does not exist, then it is created. If the schema already exists, and it is identical (this is a stricter check than compatibility; all columns must be present in the same order with the same properties, and the same data indexes must be defined), then the method returns False. If the schema already exists and is not identical, then an error is thrown. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param partition_column_name: name of the partitioning column :type partition_column_name: str :param prototype: table with definition to derive schema from :type prototype: Table :param data_indexes: data indexes to create in the table schema, each expressed as the list of column names comprising the index. Defaults to None. :type data_indexes: Optional[list[list[str]]] :returns: True if the partitioned table schema was added, False if it already existed :raises DHError: .. py:method:: add_table_partition(namespace, table_name, partition_column_value, table) Adds a single partition data of the provided partition column value to a partitioned, directly manipulated user table. The partition mustn't already exist. The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column. Note, if an error occurs during writing, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param partition_column_value: value for the partitioning column, e.g. "2015-09-25" for "Date" :type partition_column_value: str :param table: table to write data from :type table: Table :raises DHError: .. py:method:: add_unpartitioned_table(namespace, table_name, table, data_indexes = None) Adds a unpartitioned, directly manipulated user table in the database. Writes an unpartitioned user table to disk. If the namespace does not exist, then it is created. Also write out the definition of the table as the schema. The schema must not already exist. Note, if an error occurs during writing, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param table: table that the definition and data will be based on :type table: Table :param data_indexes: data indexes to create in the table schema, each expressed as the list of column names comprising the index. Defaults to None. :type data_indexes: Optional[list[list[str]]] :raises DHError: .. py:method:: append_live_table(namespace, table_name, partition_column_value, table) Appends all rows from a given table to a live (centrally managed) user table partition. The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column. This method is asynchronous. After returning, the data may not be immediately available. It is possible for the write to fail after this method has returned. When multiple workers append to a partition, ordering is imposed outside the worker by other system components. Note, if an error is thrown during appending, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param partition_column_value: value for the partitioning column, e.g. "2015-09-25" for "Date" :type partition_column_value: str :param table: table to append rows from :type table: Table :raises DHError: .. py:method:: append_live_table_incremental(namespace, table_name, partition_column_value, table) Appends all rows from the given source table to a live (centrally managed) user table partition. When rows are added to the source table, they are additionally appended to the partition. The source table updates can only have additions and shifts. No modifications or removals are permitted. The data table must have a mutually compatible definition with the current schema. The data table must not have a column with the same name as the partitioning column. This method is asynchronous, after returning the data may not be immediately available. It is possible for the write to fail after this method has returned. When multiple workers append to a partition, ordering is imposed outside the worker by other system components. A reference must be maintained to the returned AutoClosable object to ensure expected functionality; calling close() on it will stop incremental appends, and clean up related resources. Note, if an error is thrown during appending, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param partition_column_value: value for the partitioning column, e.g. "2015-09-25" for "Date" :type partition_column_value: str :param table: table to append updates from :type table: Table :returns: the object used to ensure and stop expected functionality :raises DHError: .. py:method:: catalog_table() Returns a table of the available tables. The result table contains 3 string type columns: Namespace, TableName, and NamespaceSet which is either System or User. :returns: a table .. py:method:: delete_input_table(namespace, table_name) Deletes the input table given the namespace and table name. :param namespace: the namespace of the input table :type namespace: str :param table_name: the name of the input table :type table_name: str :returns: True if the input table was deleted, False if there was no data or preexisting schema for the input table :raises DHError: .. py:method:: delete_live_table_partition(namespace, table_name, partition_column_value) Deletes a partition from a live (centrally managed) user table. Note, if an error is thrown during deleting, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param partition_column_value: value for the partitioning column, e.g. "2015-09-25" for "Date" :type partition_column_value: str :returns: True if the partition was deleted, False if there was no partition or preexisting schema :raises DHError: .. py:method:: delete_partitioned_table(namespace, table_name) Deletes all partitions, whether direct manipulated or live (centrally managed), and the schema, from a partitioned user table. All partitions from the table are deleted sequentially. If a partition cannot be deleted, then the operation fails but some data may have already been removed. After all partitions are deleted, then the schema is deleted. If the schema does not exist, then data is not deleted. If there is no data, but the schema exists, then the schema is deleted. The namespace is not removed, even if this is the last table in the namespace. Note, if an error is thrown during deleting, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :returns: True if data was deleted, False if there was no data or preexisting schema :raises DHError: .. py:method:: delete_table_partition(namespace, table_name, partition_column_value) Deletes the partition of the provided partition column value from a partitioned, directly manipulated user table. Note, if an error is thrown while deleting, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param partition_column_value: value for the partitioning column, e.g. "2015-09-25" for "Date" :type partition_column_value: str :returns: True if the partition was deleted, False if there was no partition or preexisting schema .. py:method:: delete_unpartitioned_table(namespace, table_name) Deletes a unpartitioned, directly manipulated user table and its associated schema. If the schema does not exist, then data is not deleted. If there is no data but the schema exists, then the schema is deleted. The namespace is not removed even if this is the last table in the namespace. Note, if an error occurs during deleting, the state of the table is undefined. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :returns: True if the data was deleted, False if there was no data or preexisting schema :raises DHError: .. py:method:: has_table(namespace, table_name) Identifies if the table is defined in the Database. and visible to the user. :param namespace: the namespace :param table_name: the table :return: True if the schema exists and is visible to the user, else False .. py:method:: historical_partitioned_table(namespace, table_name) Retrieves the specified historical table as a partitioned table from the Database. The result's key columns will be derived from the table's partitioning columns as well as any internal partitions. :param namespace: the namespace :type namespace: str :param table_name: the table name :type table_name: str :returns: a PartitionedTable object .. py:method:: historical_table(namespace, table_name, internal_partition_column = None) Retrieves a historical Table for the specified namespace and table name. :param namespace: the namespace :type namespace: str :param table_name: the table name :type table_name: str :param internal_partition_column: Set the name of the internal partition column, defaults to None, meaning no internal partition column is included :type internal_partition_column: str :returns: a Table object .. py:method:: input_table(namespace, table_name) Retrieves the specified input table view. :param namespace: the namespace in which the table exists :type namespace: str :param table_name: the name of the table in the namespace :type table_name: str :returns: a InputTable object :raises DHError: .. py:method:: input_table_spec_for(namespace, table_name) Retrieves the current InputTableSpec for the given namespace and table. :param namespace: the namespace of the input table :type namespace: str :param table_name: the name of the input table :type table_name: str :returns: a InputTableSpec object :raises DHError: .. py:method:: live_partitioned_table(namespace, table_name) Retrieves the specified live table as a partitioned table from the Database. The result's key columns will be derived from the table's partitioning columns as well as any internal partitions. :param namespace: the namespace :type namespace: str :param table_name: the table name :type table_name: str :returns: a PartitionedTable object .. py:method:: live_table(namespace, table_name, is_refreshing = None, is_blink = False, internal_partition_column = None) Retrieves a live Table for the specified namespace and table name. :param namespace: the namespace :type namespace: str :param table_name: the table name :type table_name: str :param is_refreshing: True if the returned table should be refreshing, defaults to None, meaning to use the value of JVM property RunAndDoneSetupQuery.liveTables which defaults to False :type is_refreshing: bool :param is_blink: True if the returned table should be a blink table, defaults to False :type is_blink: bool :param internal_partition_column: Set the name of the internal partition column, defaults to None, meaning no internal partition column is included :type internal_partition_column: str :returns: a Table object .. py:method:: namespaces() Returns the list of namespaces in this database. :returns: the list of namespaces .. py:method:: table_definition(namespace, table_name) Returns a Table containing the column definitions for the table with the specified namespace and table name. :param namespace: the namespace :type namespace: str :param table_name: the table name :type table_name: str :returns: a Table object .. py:method:: table_names(namespace) Returns the list tables within a namespace. :param namespace: the namespace :type namespace: str :returns: the list of table names within namespace .. py:method:: update_input_table_schema(namespace, table_name, input_table_spec) Updates an existing input table schema using the provided InputTableSpec. Retrieve and use a new InputTable via input_table after updating an input table's specification to ensure proper behavior. When the requested update is invalid, e.g. an input table already exists but with a different spec, an exception will be raised. :param namespace: the namespace of the input table :type namespace: str :param table_name: the name of the input table :type table_name: str :param input_table_spec: the new specification for the input table :type input_table_spec: InputTableSpec :returns: True if the input table schema was updated, False if the input table already exists with the same spec :raises DHError: .. py:method:: update_partitioned_table_schema(namespace, table_name, prototype, data_indexes = None) Updates the schema of a preexisting partitioned, directly manipulated user table. If the schema does not exist an error is thrown. Not all schema modifications are permitted. The partitioning column may not be changed. Existing columns may not have their type changed. Columns may be added or deleted. Note that no data is modified by this operation. Removed columns remain on persistent storage, and added columns are treated as null on read. Note also that although each modification in isolation is verified for safety, a sequence of modifications to the schema may be unsafe. For example, deleting a column and adding it back with a new type results in unreadable data. :param namespace: table namespace :type namespace: str :param table_name: table name :type table_name: str :param prototype: table with definition to derive schema from :type prototype: Table :param data_indexes: data indexes to create in the table schema, each expressed as the list of column names comprising the index. Defaults to None. :type data_indexes: Optional[list[list[str]]] :returns: True if the partitioned table definition was updated, False if there was already an identical definition :raises DHError: