Query Monitor

The Query Monitor table contains information about all of the persistent queries you are authorized to view in Deephaven. To open a Query Monitor, click the New button, then the New Query Monitor button.

In the Query Monitor, you can view, manage, and create persistent queries. The three buttons at the top allow users to create new queries, and start or stop existing queries. The table below stores information about each query. The panel on the right displays summary information about the highlighted query, as well as tabs for configuration options and the query's script.

Query Monitors persist in the All Dashboard menu, and will retain any filters or sorts applied to the Query Monitor table, as well as any Markdown Widgets associated with the panel.

To rename, close, or make an identical copy (including applied filters and sorts) of the Query Monitor panel, right-click its tab, as shown below:

Query Monitor Table

The Query Monitor table provides information about each persistent query you are authorized to view, including the owner's name, the query name, whether it is enabled, its status and type, etc. The content in the table can be sorted and filtered like any other table. You can sort through these queries by clicking on the column name - once to sort in ascending order, twice to sort in descending order. Right-clicking on a column header will open the Deephaven header context menu, with options to hide, filter, or sort columns. Right-clicking on an individual cell opens the Deephaven cell data context menu, which allows you to filter the table by that cell's value, or copy data.

Query Summary/Configuration

When a query name is selected in the table, the panel on the right side of the Query Monitor will reveal further information about that query. This information is presented in the following three tabs: Summary, Settings, and Script. Note: if you are not authorized to edit the query, only the query summary will be available. (Permissions and Scheduling tabs will be available in the next release.)

Summary

In the example below, the Summary tab displays details for the highlighted query named "DemoQuery", presenting the same information as in the Query Monitor table one one easy to read screen. Details include the query's start and end time, the last time the query was updated or modified, worker settings, scheduling information, its permissions, as well as full exception details when applicable.

Settings

The Settings tab opens the Persistent Query Configuration Editor. This displays configuration details about the highlighted query. Users with the appropriate permissions can edit the configuration settings for existing queries in this window. See Creating Persistent Queries for more information.

Script

The Settings tab opens the Persistent Query Script Editor. This displays the script of the highlighted query, and may be edited in this window. See Creating Persistent Queries for more information.

Creating Persistent Queries on the Web

To create a new Persistent Query, select the New button at the top of the Query Monitor.

This opens a new instance of the Persistent Query Configuration Editor.

Settings

The Settings tab includes the following configuration fields:

Enabled - This button indicates whether the query is enabled. When Enabled is set to "On", the query will attempt to run according to its schedule. If you do not want the query to run, toggle the button to "Off".

Name- Query names can be any combination of characters and numbers. However, each query in a given installation of Deephaven must have a unique name.

Owner - The author of the query. When creating a new query, your username is automatically added as the owner for that query. Note: This field can be edited by superusers only.

Type - This is the query configuration type. At this time, only the LiveQuery configuration type is available in Deephaven on the web.

DB Server - The DB Server setting allows you to select the database server associated with your installation of Deephaven and the chosen query configuration type.

Server classes determine what types of queries can be run on each server, and are used by the console to determine which database servers a user can select for each query type.

A Deephaven installation has two server classes configured by default:

Merge - this class is for servers and queries that need to load data into the database, either intraday (such as the import queries), or historical (the merge query).
Query - this class is for queries that query data but do not need to write it, such as scripts.

When creating or editing a persistent query, the Persistent Query Configuration Editor will automatically populate the DB server field with appropriate options: e.g., "Query_1" for Live Query script queries.

Heap Size - How much memory (in GB) to provision for the query.

Show Advanced - This opens advanced settings for the query, as shown below:

Data Memory Ratio - Data Memory Ratio is a memory tuning parameter that specifies what percent of the memory heap is reserved for caching data read from persistent storage. This enables Deephaven to cache data frames into RAM on an as-needed basis, which in turn speeds up the processing. The default value of ".25" in that field means 25% of the memory noted in the setting for Memory (Heap) Usageis allocated to the database buffer cache.
JVM Profile - The Java Virtual Machine (JVM) contains a garbage collector to automatically free unreachable memory. There are multiple garbage collection algorithms, and many tuning parameters available for each algorithm. See: Remote Processing Profiles> for more information. The JVM Profile drop-down menu provides the following four options:

Default - use the default garbage collection parameters for your Deephaven system. This defaults to CMS GC, but the value can be changed by the Deephaven administrator.
CMS GC - use Java's CMS (Concurrent Mark Sweep) garbage collection
G1 GC - use Java's newer G1 (Garbage First) garbage collection
GI MarkStackSize 128M - use the Custom MarkStackSize profile
None - do not use any garbage collection parameters. Desired garbage collection parameters must be manually defined in the query's "Extra JVM Args" settings (see next topic).

Log GC Details - Garbage Collection (GC) is a JVM memory management program that frees unreachable memory by getting rid of objects not being used by a Java application. When the check box is selected (the default setting), Garbage Collection information will be included in the server-side dispatcher logs.
Extra JVM Arguments - This field allows users to access different Java utilities that are not included in core Deephaven installation. For example, one may want to run a different profiler or debugging processor. These items can be included in the Deephaven configuration by typing the extra JVM arguments in this field.
Extra Environment Vars - Extra Environment Variables are used to pass additional configuration information to Deephaven.
Extra Classpaths - The Extra Classpaths field is used to tell Deephaven where to look on the file system (server) for additional class files.

At the bottom of the Settingstab are the following buttons:

Delete - This deletes the persistent query.

Copy - This copies the configuration settings and the script, which creates a draft of a new, identical query in the Query Monitor table.

Revert - This will undo any changes made since saving the settings, and restore the settings to the last saved version.

Save - This saves the current configuration settings.

Note: At this time, query permissions must be set in Deephaven Classic. To learn more, see Access Control in Query Configuration. A Permissions tab will be available in Deephaven on the web in our next release.

Script

The Script tab allows you to write a new persistent query script, or - if you have the appropriate permissions - edit an existing script.

Runtime- Choose your preferred programming language, Groovy or Python.

Use Git For Script Source - While query scripts are often written and stored within the Script Editor window in Deephaven, you can also store your query script in a Git Repository that is accessible to Deephaven. (Please consult with your system administrator to see if access to a Git Repository is possible for your installation. See also Integrating Git with Deephaven.) When checked, you can select from the drop-down menu the appropriate script in that repository to associate with this persistent query.

Revert - This will undo any changes made since the previous save, and restore the script to the last saved version of the query.

Save - This button saves the current query configuration. Note: for new queries, you will need to click the Start button at the top of the Query Monitor to begin initialization.

Last Updated: 02 March 2020 14:02 -05:00 UTC Deephaven v.1.20200121 (See other versions)