Customize the Installation

We are in the process of updating the information contained in this document. If you need assistance with the steps/processes described within, please contact Deephaven Support.

Configuration-as-Code

Configuration-as-Code is an important prerequisite to have reproducible and auditable configurations. It also makes configurations testable and easier to reuse.

This is especially important with distributed systems and micro services. Maintaining multiple services across a network of distributed systems, and keeping all their settings and configurations in sync is very hard to achieve when you have to manually configure everything by clicking through UI configuration screens.

It is therefore recommended to add the Deephaven configuration files to a version control system (VCS) such as Git. This allows updates to configuration files to be tested in a separate environment before deploying any configuration changes to your production Deephaven hosts.

Deephaven Configuration Files

The Deephaven configuration files are installed on the Deephaven server in:

/etc/sysconfig/illumon.d/

The following section provides a brief summary of the configuration files.

M/Monit Configurations:

/etc/sysconfig/illumon.d/monit/*.conf

Cron Configuration:

/etc/cron.d/illumon.iris.localhost

Host Configuration:

/etc/sysconfig/illumon.confs/illumon.iris.hostconfig

Deephaven Process Configurations:

/etc/sysconfig/illumon.d/resources/iris-common.prop

Authentication Configurations:

/etc/sysconfig/illumon.d/auth/keystore.authserver.pkcs12

/etc/sysconfig/illumon.d/auth/priv-authreconnect.base64.txt

/etc/sysconfig/illumon.d/auth/priv-iris.base64.txt

/etc/sysconfig/illumon.d/auth/priv-superuser.base64.txt

Schema Configurations:

/etc/sysconfig/illumon.d/schema/**/*.schema

/etc/sysconfig/illumon.d/schema/footer.schema

/etc/sysconfig/illumon.d/schema/header.schema

Installing Custom Classes

There are two methods by which customer classes can be added to your system: adding custom classes to the customer directory, or using the class push list. When Deephaven launches, these classes, such as custom widgets, will be available to end users to use in the console.

Customer Directory

To make custom classes available in Deephaven, they must be archived into a JAR file (see "Creating JAR Files" below) and then added to /etc/sysconfig/illumon.d/java_lib.

After restarting the Client Update Service, the files from the server will automatically be downloaded to the client when the Deephaven Launcher runs and be usable in the console.

Adding JAR Directories

Instead of copying JARs into /etc/sysconfig/illumon.d/java_lib, the Deephaven installation can also look for JAR files in existing customer directories. This is done by exporting the environment variable CUSTOMER_JAR_DIR with a comma-delimited list of directories in which JAR files may be contained. Any JAR files found in these directories will be added to the classpath. This is best done in the host configuration file. For example:

export CUSTOMER_JAR_DIR=/home/product1/java_lib/,/home/user/user1/java_lib

Class Push List

The class push list method is primarily intended for use by developers who are in the process of creating new classes for the Deephaven installation, or classes that only a small subset of users should have access to. For general, production use of custom classes, creating JAR files (see explanation below) for them and placing those JARs in the customer JAR directory is the recommended approach.

To utilize the class push list, a property must be defined for the console: RemoteQueryClient.defaultClassPushList=<listName>.txt. Upon console-worker connection, the specified text file is searched for on the client classpath. The file should contain fully qualified names of your classes, which should also be on the client classpath. These classes will be pushed to the server and ultimately to the associated worker, which will then be able to utilize custom classes from your client.

Creating JAR Files

Compilation

First, compile your Java code:

javac -classpath "/path/to/iris/java_lib/*" -d /destination/path CustomClass.java...

  • -classpath "/path/to/iris/java_lib/*" - ensures that you are compiling your code against Deephaven dependencies.
  • -d /destination/path - generates the appropriate package-associated directories to the destination path upon compilation.
  • CustomClass.java... - the source code files to compile.

Archiving

Then, group your compiled code into Java archive format, .jar files:

jar cf CustomClasses.jar FileToInclude…

  • cf CustomClasses.jar - creates a JAR file with the specified name.
  • FileToInclude… - can specify directories, which will in turn specify directory contents recursively.

Client Update Service Configuration for the Deephaven Launcher

The Deephaven console may be viewed over VNC without any additional client software. However, a more common implementation is performed where users access the Deephaven console through an application called the Deephaven Launcher, which must be installed on individual users' computers.

The Deephaven Launcher uses the "Client Update Service" to access code and configuration settings from your Deephaven server.  However, before installing and running the Launcher, the "Client Update Service" must be configured and enabled on the server using M/Monit.  Follow these steps to enable the "Client Update Service":

  1. On the Deephaven Server, edit the /var/lib/iris/cus/iris/getdown.txt.pre file:

Set the appbase value, replacing WEBHOST with the host address of your Deephaven Service.  For example:

#appbase = http://WEBHOST/iris/

appbase = http://10.20.30.40/iris/

  1. In the M/Monit config folder, remove the .disabled extension from the Client Update Service config file name and run monit reload. This will instruct the M/Monit daemon to reread its configuration and re-initialize.

cd /etc/sysconfig/illumon.d/monit

mv 04-cus.conf.disabled 04-cus.conf

monit reload

  1. Check the status of the getdown service:

monit status client_update_service

  1. Once the  "Client Update Service" is up and running, you can proceed to install and run the Launcher on client desktops. The installers for Windows, Mac and Linux desktops can be downloaded from the "Client Update Service" on your Deephaven Server at:

http://<IRIS_SERVER_ADDRESS>:80

Deephaven Process Ports

Deephaven processes listen on various TCP ports and port ranges. All ports are configurable.

TCP ports can be configured in the iris-common file:

/etc/sysconfig/illumon.d/resources/iris-common.prop

The default Deephaven ports and port ranges follow:

TCP Ports

Component

Process Name

Property

22013
22012
8084

Remote Query Dispatcher (RQD)

db_query_server

RemoteQueryDispatcherParameters.queryPort=22013

RemoteQueryDispatcher.workerPort=22012

RemoteQueryDispatcher.webPort=8084

30002
30003
8085

Remote Merge Dispatcher (RMD)

db_merge_server

RemoteQueryDispatcherParameters.queryPort=30002

RemoteQueryDispatcher.workerPort=30003

RemoteQueryDispatcher.webPort=8085

23000-24999

RQD Workers

RemoteQueryDispatcher_worker_<number>

RemoteQueryDispatcher.workerServerPorts=23000-23999

RemoteQueryDispatcher.workerServerWebsocketPorts=24000-24999

32000-32999

25000-25999

RMD Workers

RemoteQueryDispatcher_worker_<number>

RemoteQueryDispatcher.workerServerPorts=32000-32999

RemoteQueryDispatcher.workerServerWebsocketPorts=25000-25999

22021
22015

 

Data Import Server

db_dis

routing_service.yml:

dataImportServers:
  db_dis:
    tailerPort: 22021
    tableDataPort: 22015

22020

Log Aggregator Service

log_aggregator_service

routing_service.yml:

logAggregatorServers:
  - log_aggregator_service:
      port: 22020

22014

Local Data Table

db_ltds

routing_service.yml:

tableDataServices:
  db_ltds:
      port: 22014

22016

Table Data Cache Proxy

db_tdcp

routing_service.yml:

tableDataServices:  

   db_tdcp:      

      port: 22016

22023

Configuration Server (Centralized Schema Service, Data Routing Service and Configuration Service)

configuration_server

configuration.server.port=22023

9030,
9031

User Authentication Server

authentication_server

authentication.server.port.plaintext=9030

authentication.server.port.ssl=9031

9040
9041

User Access Control Server

db_acl_write_server

dbaclwriter.port=9040

dbaclwriter.ssl.port=9041

20126

 

Persistent Query Controller

iris_controller

PersistentQueryController.port=20126

22021 (outbound)

Log Tailer

tailer1..tailerN

This is determined by the db_dis section (see above).

80/443

Client Update Server

client_update_service

client-update-service.conf:

  server.port = 80

8123

Web API Service

web_api_service

Webapi.server.port=8123

3306

MySQL Database

mariadb_server

/etc/my.cnf:

  port=3306 # default

2812

M/Monit Daemon

monit

/etc/monitrc:

  set httpd port 2812

Last Updated: 16 February 2021 18:07 -04:00 UTC    Deephaven v.1.20200928  (See other versions)

Deephaven Documentation     Copyright 2016-2020  Deephaven Data Labs, LLC     All Rights Reserved