deephaven.perfmon

Tools to obtain internal, Deephaven logs as tables, and tools to analyze the performance of the Deephaven system and Deephaven queries.

ancestor_dot(ids, update_perf_log, ancestors_log)[source]

Returns a graphviz DOT representing Deephaven update performance ancestor data.

Parameters:

  • ids (Union[List[int], int]) – the Performance entry identifier or identifiers (EntryId) to generate the graph for

  • update_perf_log (Table) – the UpdatePerformanceLog Table

  • ancestors_log (Table) – the UpdatePerformanceAncestorsLog Table

Return type:

str

Returns

a string of graphviz DOT format data

Raises:

DHError –

ancestor_image(ids, update_perf_log, ancestors_log)[source]

Returns a deephaven.ui component with an embedded SVG image containing the hierarchy derived from the passed in UpdatePerformanceLog and UpdatePerformanceAncestorsLog for the provided Performance Entry identifier. This can be used to help understand the structure of a query.

Note that the deephaven-plugin-ui package must be installed to use this function.

Parameters:

  • ids (Union[List[int], int]) – the Performance entry identifier or identifiers (EntryId) to generate the graph for

  • update_perf_log (Table) – the UpdatePerformanceLog Table

  • ancestors_log (Table) – the UpdatePerformanceAncestorsLog Table

Return type:

deephaven.ui.Element

Returns

a UI component with an embedded graph of ancestors

Raises:

DHError –

ancestor_svg(ids, update_perf_log, ancestors_log, filename=None)[source]

Returns the contents of an SVG image containing a graph of the ancestor hierarchy derived from the passed in UpdatePerformanceLog and UpdatePerformanceAncestorsLog for the provided Performance Entry identifier. This can be used to help understand the structure of a query.

Parameters:

  • ids (Union[List[int], int]) – the Performance entry identifier or identifiers (EntryId) to generate the graph for

  • update_perf_log (Table) – the UpdatePerformanceLog Table

  • ancestors_log (Table) – the UpdatePerformanceAncestorsLog Table

  • filename (str) – the name of the output SVG file or None to not write the file, default is None

Return type:

str

Returns

the contents of an SVG image

Raises:

DHError –

metrics_get_counters()[source]

Gets Deephaven performance counter metrics.

Return type:

str

Returns:

a string of the Deephaven performance counter metrics.

metrics_reset_counters()[source]

Resets Deephaven performance counter metrics.

Return type:

None

process_info(proc_id, proc_type, key)[source]

Gets the information for a process.

Parameters:

  • proc_id (str) – the process id

  • proc_type (str) – the process type

  • key (str) – the key of the process property

Return type:

str

Returns:

a string of process information

Raises:

DHError –

process_info_log()[source]

Returns a static table with process information for the current Deephaven engine process.

Return type:

Table

Returns:

a Table

Raises:

DHError –

process_metrics_log()[source]

Returns a table with metrics collected for the current Deephaven engine process.

Return type:

Table

Returns:

a Table

Raises:

DHError –

query_operation_performance(eval_number)[source]

Takes in a query evaluation number and returns a view for that query’s individual operation’s performance data.

You can obtain query evaluation numbers, which uniquely identify a query and its subqueries, via the performance data tables obtained from calling query_performance_log() or query_operation_performance_log()

The query operation performance log contains data on how long each individual operation of a query (where(), update(), naturalJoin(), etc., as well as internal functions) takes to execute, and the change in resource consumption while each was executing.

Parameters:

eval_number (int) – the evaluation number

Return type:

Table

Returns:

a table of query operation performance data

Raises:

DHError –

query_operation_performance_log()[source]

Returns a table with Deephaven performance data for individual subqueries. Performance data for the entire query is available from calling ‘query_performance_log’.

Return type:

Table

Returns:

a Table

Raises:

DHError –

query_operation_performance_tree_table()[source]

Returns a tree table with Deephaven performance data for individual subqueries.

Return type:

TreeTable

Returns:

a TreeTable

Raises:

DHError –

query_performance(eval_number)[source]

Takes in a query evaluation number and returns a view for that query’s performance data.

You can obtain query evaluation numbers, which uniquely identify a query and its subqueries, via the performance data tables obtained from calling query_performance_log() or query_operation_performance_log()

The query performance log contains data on how long each query takes to run. Examples of what constitutes one individual query, for performance logging purposes, include:

  • A new command in the console (i.e. type something, then press the return key)

  • A sort, filter, or custom column generated by a UI

  • A call from a client API external application

Parameters:

eval_number (int) – the evaluation number

Return type:

Table

Returns:

a Table of query performance data

Raises:

DHError –

query_performance_log()[source]

Returns a table with Deephaven query performance data. Performance data for individual sub-operations is available from calling query_operation_performance_log.

Return type:

Table

Returns:

a Table

Raises:

DHError –

query_performance_tree_table()[source]

Returns a tree table with Deephaven query performance data. Performance data for individual sub-operations as a tree table is available from calling query_operation_performance_tree_table.

Return type:

TreeTable

Returns:

a TreeTable

Raises:

DHError –

query_update_performance(eval_number)[source]

Takes in a query evaluation number and returns a view for that query’s update performance data.

You can obtain query evaluation numbers, which uniquely identify a query and its subqueries, via the performance data tables obtained from calling query_performance_log() or query_operation_performance_log()

Parameters:

eval_number (int) – the evaluation number

Return type:

Table

Returns:

a Table of query update performance data

Raises:

DHError –

query_update_performance_map(eval_number)[source]

Creates multiple tables with performance data for a given query identified by an evaluation number. The tables are returned in a map with the following String keys: ‘QueryUpdatePerformance’, ‘UpdateWorst’, ‘WorstInterval’, ‘UpdateMostRecent’, ‘UpdateAggregate’, ‘UpdateSummaryStats’.

Parameters:

eval_number (int) – the evaluation number

Return type:

Dict[str, Table]

Returns:

a dict

Raises:

DHError –

server_state()[source]

Returns a table of basic memory, update graph processor, and GC stats for the current engine process, sampled on a periodic basis.

Return type:

Table

Returns:

a table

server_state_log()[source]

Returns a table with memory utilization, update graph processor and garbage collection stats sampled on a periodic basis.

Return type:

Table

Returns:

a Table

Raises:

DHError –

update_performance_ancestors_log()[source]

Returns a table with Deephaven update performance ancestor data.

Return type:

Table

Returns

a Table

Raises:

DHError –

update_performance_log()[source]

Returns a table with Deephaven update performance data.

Return type:

Table

Returns

a Table

Raises:

DHError –