deephaven.ExportTools.JdbcLogger¶

A utility that logs Deephaven table snapshots and/or incremental updates to a JDBC table. It is expected that the: target table is “compatible” with the source: JDBC metadata is interrogated to ensure the types of the source table columns match those of the corresponding target columns. Column names may be mapped, but any additional conversion must be performed in Deephaven prior to logging.
This logger may be used in two “modes”. The default reproduces the content of the source table via a combination of: SQL INSERT, UPDATE and DELETE statements (batched when possible). In “log mode”, each incremental update to the source table is logged as an INSERT to the target table, along with (optional) information about which row was affected and the relevant operation (Added/Removed/Modified).

In addition, a number of different transaction modes may be used:

When “RowByRow” is specified, each logged row/operation is individually committed.
When “Atomic” is specified, each update to the source table is logged as an atomic operation - i.e. if an update affecting 100 rows is received by the table listener, these are committed as a unit.

The following modes only apply when logging snapshots:

When “Start” or “None” is specified, SQL operations are executed in the current transaction, but no commit is executed.
When “End” is specified, the final row of the snapshot is followed by a single commit operation.

The “Atomic” mode is recommended in most cases for optimal performance.

class JdbcLoggerBuilder(*args, **kwargs)¶

batchSize(batchSize)¶

Specify the batch size when writing to the JDBC data source. For efficiency, the JDBC logger “batches” SQL

statements when possible. Each Deephaven table update results in at least one commit, regardless of the batch size, so this represents a maximum. The default batch size is 500 rows.

Batching is only effective when the TableLoggerBase.Flags setting is not “RowByRow”, since that effectively requires a commit for every row. When logging with updates, the “Atomic” setting is recommended, which requires only one commit for each Deephaven table update (which can affect any number of rows). When logging snapshots only, any setting other than “RowByRow” will take advantage of batching.

Parameters: batchSize – (int) - maximum batch size (legal range is 1 to 100,000)
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

build()¶

Creates a JdbcLogger with configuration specified by arguments to the builder.

Returns: (com.illumon.iris.export.jdbc.JdbcLogger) a new JdbcLogger

property builder¶: The java builder object.

calendar(calendar)¶

Specify the Calendar to use when logging to JDBC date/datetime/timestamp columns. This can affect the way: DateTime values are logged when the target column does not directly store an offset/timezone. See PreparedStatement.setDate(int, Date, Calendar) and PreparedStatement.setTimestamp(int, Timestamp, Calendar) (int, Date, Calendar)} for details.

Parameters: calendar – (java.util.Calendar) - Calendar to use for logging
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

dataColumn(*args)¶

Add a data column for logging.

Overload 1

param targetColumn: (java.lang.String) - column name, with the same name in the source and target.
return: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

Overload 2

param targetColumn: (java.lang.String) - column name in the JDBC table
param sourceColumn: (java.lang.String) - column name in the source Deephaven table
return: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

jdbcPassword(jdbcPassword)¶

Specify the JDBC database password. Only necessary if not specified as part of the JDBC URL.

Parameters: jdbcPassword – (java.lang.String) - JDBC database password
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

jdbcUser(jdbcUser)¶

Specify the JDBC database user. Only necessary if not specified as part of the JDBC URL.

Parameters: jdbcUser – (java.lang.String) - JDBC database user name
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

keyColumns(*keyColumns)¶

Specify the set of columns that represent the primary key when logging to JDBC. This set of column(s) will be

used in order to generate the WHERE clause in UPDATE and DELETE operations, when not in “log mode”. If in log mode, key columns are unnecessary, since every source table update will simply be logged as an INSERT to the JDBC target.

The key columns should be either composed of a subset of the data columns (as they are named in the SQL table), or the rowIndex column. Each set of unique values in these columns should represent a unique row in order for logging to work properly. If present, a rowIndex column always provides this feature.

Parameters: keyColumns – (java.lang.String…) - an array of columns to use as the composite primary key
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

logMode(logMode)¶

Set log mode indicator. If not specified, this defaults to false.

When in log mode, each update to the source Deephaven table will result in an INSERT into the target table (i.e. the target will be an append-only log of every update to the source).

Otherwise, each operation is replicated in the target using the appropriate INSERT/UPDATE/DELETE operation, using the specified row key columns to uniquely identify rows.

Parameters: logMode – (boolean) - if true, run in log mode
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

operationColumn(operationColumn)¶

Specify the JDBC column that will receive the operation that resulted in that row. This is most useful in: “log mode”, in order to differentiate between Added/ModifiedOld/ModifiedNew/Removed operations in the source table. A PreparedStatement.setString(int, java.lang.String) will be used to set this value, so the target column should be a VARCHAR with length 11 or more.

Parameters: operationColumn – (java.lang.String) - JDBC column that will receive the operation (must be compatible with a string value)
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

rowIndexColumn(rowIndexColumn)¶

Specify the JDBC column that will receive the rowIndex of the source table row that resulted in a given: JDBC row. A PreparedStatement.setLong(int, long) will be used to set this value, so the target column should be an SQL BIGINT or equivalent.

Parameters: rowIndexColumn – (java.lang.String) - JDBC column that will receive the operation (must be compatible with 64 bit integer)
Returns: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) this builder

builder(*args)¶

Creates a JdbcLogger JdbcLogger.Builder.

Overload 1

param logger: (com.fishlib.io.logger.Logger) - logger to use when initializing
param jdbcDriver: (java.lang.String) - JDBC driver class name
param jdbcUrl: (java.lang.String) - JDBC connection URL
param catalog: (java.lang.String) - JDBC database catalog
param schema: (java.lang.String) - JDBC database schema
param tableName: (java.lang.String) - JDBC database table name
return: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) JdbcLogger builder.

Overload 2

param logger: (com.fishlib.io.logger.Logger) - logger to use when initializing
param jdbcDriver: (java.lang.String) - JDBC driver class name
param jdbcUrl: (java.lang.String) - JDBC connection URL
param schema: (java.lang.String) - JDBC database schema
param tableName: (java.lang.String) - JDBC database table name
return: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) JdbcLogger builder.

Overload 3

param logger: (com.fishlib.io.logger.Logger) - logger to use when initializing
param jdbcDriver: (java.lang.String) - JDBC driver class name
param jdbcUrl: (java.lang.String) - JDBC connection URL
param tableName: (java.lang.String) - JDBC database table name
return: (com.illumon.iris.export.jdbc.JdbcLogger.Builder) JdbcLogger builder.

deephaven.ExportTools.JdbcLogger¶

Deephaven

Navigation

Related Topics