Deephaven Quick Start Guide

The Quick Start Guide is designed to be a very short exercise to familiarize new users with the Deephaven interface, provide examples of using the Deephaven query language to generate tables and plots, save queries, and build a customized workspace.

Prerequisites

Deephaven needs to be installed for your enterprise.
To use Deephaven Classic, the Deephaven Launcher application needs to be installed on your PC, with an instance configured to access the stored data.

Please consult with your system administrator if assistance is needed.

Open Deephaven

Working with Dashboards

Note: This step applies to Deephaven on the web only.

Dashboards are customized collections of content including tables, plots and/or sets of nested panels. You can manipulate tables and plots in dashboards by dragging and dropping, as you’d expect. Using, creating, and sharing dashboards is fundamental to the value of the experience.

Step 1 - Create a Dashboard

Let's create our own workspace. Select New Dashboard to open an empty dashboard window called "Untitled".

Right-click the tab to rename your dashboard - we'll name ours "Demo Dashboard".

Step 2 - Add Content

You can add content to your dashboard using the Panels menu. This menu presents a list of Persistent Queries available to you. Each of these generate tables and/or plots that you can add to your dashboard(s) or consume via APIs.

For example, select the query named "LearnDeephavenPQ", and then select the name of the table or plot from the list to add it to your dashboard layout. In this case, we'll select the "StockQuotes" table.

The table will then be added to the dashboard.

You can repeat this workflow to add as many tables and plots to the dashboard as you'd like, including those from different Persistent Queries.

Go back to the Panels menu and add a plot called "RetailPlot" to the Dashboard. This is a OneClick plot, and we'll see what it can do in the next step.

Step 3 - Manipulate the Data

As you can see when you added RetailPlot to your Dashboard, Deephaven prompts you to add a filter in order for the plot to draw. We've built instructions like these into the interface to help you learn how to use Deephaven as you explore and try your own analyses.

Click the blue Add Input Filters button and an Input Filter for the appropriate column will be added to your workspace (in this case, USym as configured in the underlying query. More on that later). Type a USym value - e.g., GOOG or AAPL - and not only will the plot draw those values, but the StockQuotes table open in your dashboard will also filter on the USym column accordingly.

What if you want to filter the table only? Right-click any column header to check out additional filter and sorting options.

For now, choose Quick Filters to open the Quick Filters bar above the StockQuotes table's column headers. You'll see a field above each column - type a value to instantly filter that column, as shown below:

Don't know the values off the top of your head, or you want finer grained control? Click the funnel icon in the Quick Filters bar to pull up the Advanced Filters.

The Advanced Filters dialog provides more options for the filter type, and also auto-populates the column values, allowing you to search or scroll through the options to select one or multiple values.

Step 4 - Make a Plot

In Step 2, we opened a plot from the Panels menu. You can also make plots directly from the Chart Builder feature.

In the table containing the data you want to plot, click the hamburger icon in the upper right corner.

This will open the Table Options menu. Select Chart Builder.

The Chart Builder interface allows you to choose one of the five available chart types, and configure the applicable axis values. For this example, choose Timestamp as the X-Axis value and BidSize as the Series value.

Click Create and the new plot will open in its own tab next to RetailPlot.

Want to move these panels around? Move on to Step 5.

Step 5 - Customize Your Workspace

The tables and plots open in default positions, but you can move and/or resize any panel.

For instance, if you wanted BidSize by Timestamp to be larger, you can use your cursor to expand the panel size, or drag-and-drop panels to new locations. You could move your Input Filter next to StockQuotes and shrink its size so it doesn't take up so much real estate, as shown below:

Your layout retains your changes when you close it. To re-open your Dashboard or another existing Dashboard, go back to the New+ tab and choose an item from the All Dashboards list.

Working with Code Studios and Queries

A Code Studio is a workspace for writing queries and code to interrogate and manipulate data. Multiple Code Studios may be open simultaneously, including those using different programming languages. This is especially beneficial if you have a variety of independent, ad hoc analyses or development projects underway at the same time.

Note: These steps can be performed in Deephaven Classic or Deephaven on the web.

Step 1 - Launch a Code Studio

Navigate to the Code Studio tab on the left of your navigation bar:

(You can also launch a new Code Studio from the +New tab by selecting the New Code Studio button.)

Click the Connect button. The default settings will be fine for this exercise. We'll be using Python in these examples.

Step 2 - Access Your Data

Deephaven uses data stored in tables. These tables are organized by namespaces. This is analogous to the way you store information on your computer. Just as you might use folders (or directories) to hold individual files, Deephaven uses namespaces to hold individual tables.

You can use the Deephaven Query Language to write scripts that will immediately render tables and plots in the console. This real-time feedback loop (on all sorts of data) is great for on-the-fly analysis, debugging, and development.

To access tables in Deephaven, you need to know:

where the table is located (the name of the namespace), and
the name of the table

Let's see how the Code Studio works by writing a simple query. For this exercise we are going to access the namespace called "LearnDeephaven", and open a table called "StockTrades". (Note: This configuration should be included with the default Deephaven installation. Contact your system administrator if it is not.)

Either copy and paste or type the following query immediately after the flashing cursor in the Console section of your new Code Studio:

table1 = db.t("LearnDeephaven","StockTrades")

This is a Deephaven query. Go to our Writing Queries section for more details about the Deephaven Query Language, but the following diagram summarizes what the query is asking Deephaven to do:

Press Enter or Return on your keyboard to run the query in Deephaven.

A new tabbed panel named table1 will appear in your workspace:

The table you just accessed is partitioned. This means the table is organized and grouped into different smaller segments. By organizing a table like this, the computing power needed for analysis can be applied more efficiently, especially when your table has millions or billions of rows of data.

In this case, the table is partitioned by the date shown in the first column. Since the table is large, Deephaven opens the table to a single partition (2017-08-25). You can choose Ignore & Fetch All to load the entire table, or select a different partition:

For this exercise, click Append Query at the right to update the query to automatically filter on that date. This will reload table1 with a filter (using the .where method) on the Date column as visible in your console.

Step 3 - Manipulate the Data

Now that we have a table loaded into the Deephaven console, we can further manipulate the data to serve our needs. For example, we can filter the data to include only certain categories, values or other aspects. We can add new columns or remove columns we don't need. Or, we can sort, group or aggregate the data, or join the data in this table with data from another table. The Deephaven Query Language provides many methods for manipulating the data further. However, since we're just getting started, we'll stick with something relatively simple.

The table now holds quantitative information about the individual trades for each of 10 different financial securities (AAPL, AXP, BAC, CSCO, GOOG, IBM, INTC, MSFT, PFE, XOM). If you scroll in the window holding the table, you can start to get a sense of the large volume of data the table holds. When you clicked Append Query, a new query with a filter applied was produced to show only the data for August 25, 2017.

table1=table1.where("Date=`2017-08-25`")

We can reduce the dataset still further, using an additional .where filter. Copy and paste, or re-type, the following query into the Console:

table2=table1.where("Date=`2017-08-25`" , "Sym = `XOM`")

The following diagram illustrates what you are asking Deephaven to do:

When this query is run in Deephaven, a new tab for table2 appears. If you scroll through the table, you'll see it only contains the data for when the value XOM is contained in the column titled Sym. The data has now been reduced to only 38,387 rows (hover over any column header to generate these statistics):

You can accomplish the same result by using shortcuts within the user interface, as we did when working with Dashboards. Right-click any column header or within the table data to pull up those options.

Column Header Context Menu	Table Data Context Menu (Note: initiated from a XOM cell in the Sym column)

Step 4 - Make a Plot

We showed you how to make a plot using the Chart Builder when working in Dashboards. The Table Options menu is available in Code Studios, but you can also use the Deephaven Query Language.

In the previous step, we filtered the table to show only one day's worth of data for only one ticker symbol from the Sym column. Let's plot that data to more easily visualize any trend or pattern it may show. Here's the query to plot the price of that ticker symbol over the course of that one day:

from deephaven import Plot
plotXOM = Plot.plot("ExxonMobil", table2, "Timestamp", "Last").show()

The following diagram summarizes the components of this query:

Copy the query above and then paste it into the Deephaven.

After this query runs in Deephaven, a new panel titled plotXOM appears in the lower part of the console window. This panel shows a plot that charts the variation in the price of ExxonMobil stock over the course of one day.

Step 5 - Customize your Workspace

The plot draws to the right of the two tables, but you can also drag and drop the panel to reposition the chart in your workspace, as shown below, where plotXOM is moved from its own panel to join the nested panel set:

Close any panel in your workspace by clicking the X in its tab:

If you close table1, you can easily reopen the table again in the same session by clicking the blue button with its variable name in the console:

And remember, you can also resize any panel by placing your cursor at one of its borders and dragging-and-dropping to make your adjustments. Below, we shrink the console panel:

As with Dashboards, your layout will be saved. You can also rename your Code Studio tab and access it again from the New+ button and the All Dashboards list.

Creating and Saving Persistent Queries

So far, you've accessed data, filtered the data, saved the data and made a plot. In the Code Studio, you completed those steps by using queries in the Deephaven console. What if you wanted to save those queries so you could use them later? Or maybe you want to share those queries or the tables generated by the queries with other individuals or teams within your enterprise. This is easy to do by making your query persistent - meaning the query is saved and stored.

There are a number of ways to create a persistent query, so you should review the Query Configuration section of our documentation. We'll review the primary method here.

Deephaven on the Web

First, highlight all of the content that you currently have in the Console panel, and then copy that content to your clipboard.

Next, click the +New button at the top of the Deephaven interface, and select New Query Monitor.

This window shows the names and details associated with persistent queries that have already been created and saved to Deephaven by someone in your enterprise, as demonstrated below. Here you can view details about the LearnDeephavenPQ query that built your Dashboard.

Click the +New button to open the Persistent Query Configuration editor.

In the Settings tab, configure the following:

In the Name field, enter the name you would like to use for this query (e.g., QuickStart).
For Type, select Live Query (Script) from the drop-down menu.
For DB Server, select the first option available (e.g., Query-1; server names vary by customer).
For Heap Size, keep .5. (This is the amount of RAM that will be assigned to run your persistent query.)

Then, select the Script tab. Click anywhere inside the Script Editor panel, and paste the content previously copied from the Console window.

When we originally accessed our table, we chose to append the query with a partition. Since that second line from the Console is redundant, delete it to clean up your script. Make sure to choose the appropriate programming language in the Runtime drop-down - in this case, Python.

Click Save, and you'll see your query added to the Query Monitor Table and cycle its status to "Running":

Your query is now available in the Panels menu for use in both Dashboards and Code Studios.

Deephaven Classic

First, highlight all of the content that you currently have in the Console panel, and then copy that content to your clipboard.

It should look something like the following before you highlight the content:

After copying the content, click the Query Config button at the top of the interface. A dialog window will then present the option to rename the panel, or keep the existing Query Config name.

Click OK to keep the same name and open the Query Config window. This window shows the names and some of the properties associated with persistent queries that have already been created and saved to Deephaven by someone in your enterprise, as demonstrated below.

Click the New button, which can be found in the Configuration section in top right corner of this panel. This will open the Persistent Query Configuration Editor window.

In the Name field, enter the name you would like to use for this query. (e.g., QuickStart)
In the drop-down menu to the right of Configuration Type, select Live Query (Script).
In the drop-down menu to the right of DB Server, select the first option available. (Server names vary by customer.)
In the field to the right of Memory (Heap) Usage, enter the number 4. (This is the amount of RAM that will be assigned to run your persistent query.)

Now, select the Script Editor tab. Click anywhere inside the Script Editor panel, and paste the content previously copied from the Console window.

The Script Editor window should now look very similar to the following:

We need to do a little clean-up to remove some artifacts copied from the console. At the beginning of each line is the greater-than symbol (>) and a space. Remove them from each line. Also, between each of the longer lines is the name of the table (or plot) the previous line generated. Remove those values as well.

The contents in the Script Editor window should now look like this:

In order for each table to open properly when the persistent query runs, we need to update the script ine line 1 for table1. When we attempted to open this table in the console, we were asked to fetch a selected partition, or to open all partitions. If we want all the partitions in table1 to open without filtering to a specific date, as in table2, we can add an empty .where() filter to or query, as shown below:

Click the OK button at the bottom of the window. Doing this will save your persistent query and return you to the Query Config window. Your newly created persistent query should be listed in alphabetical order. In the Status column, the value in the row for the QuickStart query should show values changing from Acquiring Worker to Initializing to Running.

To open a table associated with the QuickStart query, first highlight the row in the QueryConfig panel that contains the QuickStart query. At the bottom of the Deephaven interface, you'll see the button Show Table. To the right of that button is a drop-down menu. Click the drop-down menu and select table3 as shown below.

Then click Show Table.

A new panel opens in the Deephaven console with a tab named QuickStart.table3, as shown below.

You can also open a plot in a similar fashion. We first need to return to the list of persistent queries, so click the tab labeled Query Config at the top left portion of the console window. Then, just as you did before, click once on the row containing the QuickStart query.

At the bottom of the Deephaven interface, you'll see the button Show Widget. (Note: "Widget" is a term used to describe various user interface components. In Deephaven, a plot is a common form of widget.)

To the right of the Show Widget button is a drop-down menu. Click the drop-down menu and select plotXOM, as shown below. There is only one plot associated with this query, so there is only one option in the drop-down menu. If the query generated more than one plot, the names of all of the generated plots would be in this list.

Then click Show Widget.

A new panel opens in the Deephaven console with a tab named QuickStart.plotXOM, as shown below.

Last Updated: 16 February 2021 18:06 -04:00 UTC Deephaven v.1.20200928 (See other versions)