Deephaven Launcher and Client Configuration

Client Components Update

When a new Deephaven console is launched from the Launcher, the Launcher will check whether the instance has the latest versions of JAR files and other files from the Client Update Service (CUS) associated with the instance. It will download newer files if updates are available. Once the console is running, it will check for changes every 10 minutes. If new files are found (because an update has been applied or a configuration file changed on the server), a message banner will appear on the client console to inform the user that the console is out of date. A relaunch of the console will then be required to download and apply the new files. Continuing to run the console with out of date components may appear to work fine. However, problems may arise when client-side settings are not correct for the current environment, or when the client is using classes that no longer match the versions installed on the server.

The directory /var/www/lighttpd/iris/iris/ contains the files used to control the Client Update Service. This directory is populated by syncing files from system directories as well as etcd, and is rebuilt every time the client_update_service process is restarted; these files should never be edited. If needed, previous versions of the /var/www/lighttpd/iris directories can be found in /var/www/lighttpd/backups; these backups are created every time deephaven is upgraded, and can be safely discarded once the new version has passed testing.

The getdown.txt file on the server (/var/www/lighttpd/iris/iris/getdown.txt) controls most of the Client Update Service behavior. The getdown.txt file on the server (/var/www/lighttpd/iris/iris/getdown.txt) controls most of the Client Update Service behavior. This file is dynamically built by the Client Update Service when it starts, using files from the directory /etc/sysconfig/illumon.d/client_update_service and adding required resources from the Deephaven installation. This directory is also recreated each time the Deephaven software is installed, but existing files are copied to the new directory.

/var/www/lighttpd/iris/iris/getdown.txt is constructed by concatenating two files:

a system-owned /var/www/lighttpd/iris/iris/getdown.txt.pre, which is automatically replaced every time Deephaven is upgraded;
a customer-owned/etc/sysconfig/illumon.d/client_update_service/getdown.global, which is never replaced, and where all customer-controlled configuration should be placed.

For a comprehensive explanation of system vs. customer files, see the Client Update Service documentation. The customer-owned files are as follows:

getdown.host - this contains the host address, including an optional port
getdown.protocol - this contains the expected protocol, typically https
getdown.global - this contains all the parameters used by getdown and contains most of the parameters copied to getdown.txt
getdown.java.1.8, getdown.java.11, getdown.java.13 - these files contain default JVM parameters that will be use by clients based on the Java installation used

getdown.txt contains the settings detailed below (copied from getdown.txt.pre and getdown.global), and also a list of files to be downloaded. The contents of the generated getdown.txt file and all files to be downloaded are hashed and the values saved in digest2.txt. This is the file clients use to see whether there are updates to be applied, and to know what files to download.

When any of the files needed by the client are updated, you must restart the Client Update Service to regenerate the configuration files and digest. CUS is restarted automatically after a Deephaven product RPM is applied to a query server. The updated digest2.txt will allow running clients to see that updates are available when they next check for changes.

The script /usr/illumon/latest/cus/bin/cus is responsible for concatenating the system-owned /var/www/lighttpd/iris/iris/getdown.txt.pre with the customer-owned /etc/sysconfig/illumon.d/client_update_service/getdown.global to form the final configuration file, /var/www/lighttpd/iris/iris/getdown.txt, which is what getdown uses to decide what to send to client machines. The CUS script is also responsible for copying the client binaries and resources from the latest Deephaven release, the Deephaven customer and plugin directories as well as prop files from etcd into the web directory to be distributed to client machines.

Customer-Updateable Files

The main settings and sections in getdown.global and their purposes follow:

`appbase`

This indicates the URL from which all subsequent files will be downloaded. All other files are relative to this root. This is generated from getdown.protocol, getdown.port and getdown.host files, and should never be directly edited.

appbase = https://irisprod.company.com/iris

To change the appbase, you should instead edit getdown.protocol, getdown.host, and getdown.port, as detailed in the Client Update Service documentation.

The default page (index.html) for the parent of this path contains overview information about the Deephaven client and provides installer downloads. This page can be customized with environment-specific information, such as whom to contact for access to Deephaven.

`ui.install_error`

Certain errors will point to this page for more information. The default page provides contact information for Deephaven Data Labs support. It can be customized to add internal support information, if desired. Again, this is generated from getdown.protocol and getdown.host, and should never be directly edited. To change this value, follow the instructions in the Client Update Service documentation.

ui.install_error = https://irisprod.company.com/iris/error.html

`ui.name`

The default value is:

ui.name = Deephaven

However, users can customize this setting. For example:

ui.name = Deephaven Data Labs for Company Name Here

This will be the title of the application while downloading files.

`Configuration.rootFile`

This sets the properties file to be used by the client. The default is iris-common.prop.

jvmarg = -DConfiguration.rootFile=iris-common.prop

Non-Updateable Files

The main settings and sections in getdown.txt.pre and their purposes follow. These can not be edited.

`classpath`

Resources will be searched for in the order these directories are listed, including searching inside JARs. Most Deephaven files are in the resources and java_lib directories. There are separate settings for classpaths on Windows, Linux, and OSX.

jvmarg = -classpath

jvmarg = [windows] "%APPDIR%\private_classes;%APPDIR%\private_jars\*;%APPDIR%\override;%APPDIR%\hotfixes\*;%APPDIR%\config;%APPDIR%\resources;%APPDIR%\java_lib\*;%APPDIR%\plugins\hotfixes\*;%APPDIR%\plugins\override;%APPDIR%\plugins;%APPDIR%\plugins\*;%APPDIR%\customer_jars\*"

jvmarg = [linux] %APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override:%APPDIR%/hotfixes/*:%APPDIR%/config:%APPDIR%/resources:%APPDIR%/java_lib/*:%APPDIR%/plugins/hotfixes/*:%APPDIR%/plugins/override:%APPDIR%/plugins:%APPDIR%/plugins/*:%APPDIR%/customer_jars/*

jvmarg = [mac] %APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override:%APPDIR%/hotfixes/*:%APPDIR%/config:%APPDIR%/resources:%APPDIR%/java_lib/*:%APPDIR%/plugins/hotfixes/*:%APPDIR%/plugins/override:%APPDIR%/plugins:%APPDIR%/plugins/*:%APPDIR%/customer_jars/*

%APPDIR%/config/*:%APPDIR%/private_classes:%APPDIR%/private_jars/*:%APPDIR%/override/*:%APPDIR%/resources:%APPDIR%/java_lib/*

`clean_target`

This specifies directories under the instance directory whose contents should be completely replaced when updating. This is required for parts of the classpath that use wildcards to find JARs whose names will change from version to version.

clean_target = java_lib

<numerous clean_target entries follow>

While it is possible to add your own clean_target to getdown.global, in practice, it is very rare to need to do so. Do NOT edit targets in getdown.txt.pre, as they will be lost upon upgrading.

`Code and Resources`

Code and resource entries in getdown.txt identify specific files for which digest values should be calculated, and that the Launcher should pull down to a client. The base path for these files is /var/www/lighttpd/iris/iris. The script /usr/illumon/latest/cus/bin/cus rsyncs several server directories there prior to regenerating getdown.txt and updating the digest. The default paths are:

/usr/illumon/latest/java_lib//etc/sysconfig/illumon.d/java_lib/
/etc/sysconfig/illumon.d/override/
/etc/sysconfig/illumon.d/hotfixes/
/etc/sysconfig/illumon.d/integrations/
/etc/sysconfig/illumon.d/resources/
/etc/sysconfig/illumon.d/calendars/
/etc/sysconfig/illumon.d/chartthemes/
/etc/sysconfig/illumon.d/dh-config/

The script then goes on to exclude some items, such as configuration files not needed by client installations. It will also include all the prop files stored into etcd; if you update your props in etcd, you will need to restart the client_update_service process to have those new files sent to client machines.illumon.d.

Additional files that are needed by Launcher clients should usually be added to /etc/sysconfig/illumon.d/java_lib/ or /etc/sysconfig/illumon.d/resources/. However, this will add the files to the classpath of server processes as well. Alternatively, custom individual code entries could be added to getdown.global.

Like many other text configuration files, getdown.global, getdown.txt and getdown.txt.pre use the pound/hash symbol to indicate comment lines.

Note: jvmarg = … can also be used to configure any globally-applicable JVM arguments to be passed to the Deephaven console process.

Uploading Custom Scripts and Resources to Swing Clients

It may be required to execute a custom script on the swing clients, for example to download jars from a repository. Other resources can also be uploaded to clients. This can be done as follows.

The directory /etc/sysconfig/illumon.d/client_update_service/resources contains additional files which will be copied to the getdown directory /var/www/lighttpd/iris/iris. All this should be done as the Deephaven administrative user, typically irisadmin.

Create your additional files in the directory. For example, to create a script to be executed on the swing clients two files will be needed. The first is the script, which in this example is called prelaunch.sh:
#!/usr/bin/env bash <Script commands>
The second must be called prelaunch.txt, and contains commands to be executed by the uploading client. A shell command is specified with "exec". getdown won't create the script with executable permissions, so this must be done as a command. In this example:
exec = chmod 754 ./prelaunch.sh exec = ./prelaunch.sh
Update /etc/sysconfig/illumon.d/client_update_service/getdown.global to include the new resources.
# prelaunch file resource = prelaunch.txt resource = prelaunch.sh
Restart the CUS:
sudo monit restart client_update_service

Launcher Properties Files

When the Launcher starts a Deephaven console, a default installation will use the iris-common.prop properties file to control its settings. (See Configuration.rootFile above for details on specifying a different properties file.) This file, and all included properties files, must be exported from and re-imported into etcd. They will typically be in <Program>/<instance>/resources, but can be in other folders or inside a JAR file.

Properties files and JAR files are downloaded automatically from the Deephaven server by the Launcher, so no persistent changes can be made by editing these files on the client. To make global changes, only the iris-environment.prop file should be modified on the server. You will need to export the file from etcd, edit the file, then import the changed file into etcd:

sudo -u dbquery /usr/illumon/latest/bin/dhconfig --export iris-environment.prop /tmp/propfiles

vi /tmp/propfiles/iris-environment.prop

sudo /usr/illumon/latest/bin/dhconfig --import iris-environment.prop iris-environment.prop

Restart the CUS once the changes have been made.

To instead apply some configuration changes to the properties of specific client installations, these can be made by adding property keys and values to one of the jvmargs.txt files. There are several copies of this file that have varying levels of applicability depending on the location:

<WorkspaceRoot>/jvmargs.txt - applies to all instances
<WorkspaceRoot>/<instance_name>/jvmargs.txt - applies to all workspaces and consoles in the instance
<WorkspaceRoot>/<instance_name>/workspaces/<workspace_name>/jvmargs.txt - applies to this Deephaven workspace
<WorkspaceRootapplication_installation_path>/<instance_name>/workspaces/console/jvmargs.txt - applies to consoles in the instance

Settings from all applicable files will be combined. If there are settings that exist in more than one of the files, the last one evaluated will take precedence. They are evaluated in the order listed above. The settings in the jvmargs.txt files also override settings from properties files.

The format of jvmargs.txt is similar to other properties files in that blank lines are ignored and comments start with a pound/hash [#] symbol. However, as this file is being used to pass extra arguments into the Java command line, most of its key=value entries will start with -D. For example:

-DCsvExport.useFileSelectionDialog=true

Client Update Service Script

Timeout

In some instances, the CUS may take longer than 30 seconds to start. By default, monit ends processes after 30 seconds. To avoid this, the default timeout is 90 seconds:

check process client_update_service with pidfile /var/run/illumon/cus.pid
start program = "/etc/init.d/cus start /path/to/logs" with timeout 90 seconds
stop program = "/etc/init.d/cus stop /path/to/logs"

Note that the timeout is pure monit functionality and actually applies to all services. If needed, customers can modify the "with timeout xx seconds" argument to increase the time monit gives the process.

Logs

In addition, the CUS script now creates persistent log files. The default log location is /var/log/deephaven/misc. If a different location is desired, the path can be added to the start and stop program lines as above. These logs can be used to diagnose issues or be sent to Deephaven support.

Installation Location Information

Three types of files are stored during Deephaven installation and operation:

Launcher files - These files are installed on a per-user basis or per-machine basis, depending on the actual installation option chosen. These files do not change, and they do not contain data, downloaded JARs, or workspace files.
Program files (also known as instance files) - These files include the JARs and the configuration files downloaded and synchronized for each instance, as well as log files generated by the Launcher and Deephaven program.
Workspace files - These files are the workspace settings, backups, and history files.

The Deephaven Launcher displays the locations of Workspaces, Program, and Log files at the bottom right, as shown here:

Windows Installation

The Deephaven Installer is used for all Windows installations (See: User Guide to Installing Deephaven).

When the Deephaven Launcher is Installed with the "Per-user" option, locations for all three file types can be chosen. The user's selections for the locations of Deephaven files are saved in the registry for reuse when updating the installation. The values are also saved in the Launcher directory for use by the Launcher (which does not use the registry). File locations can also be changed at a later time by changing system properties as described below.

When the Deephaven Launcher is installed with the " All users" option, it makes less sense to specify workspace and instance locations, so the defaults are always used. These default locations are detailed individually in the section below.

System administrators may change the defaults by setting properties in the system-wide launcher properties file. This works on shared systems (e.g., Linux, Citrix, etc.) and in Windows environments where a file can be distributed to all users.

Mac/Linux Installation

The Deephaven Installer is not used on Mac or Linux machines. Instead, users extract the Launcher files from a tar file and then launch the program by running a bash script in the extraction folder. For detailed instructions, see: User Guide to Installing Deephaven.

On a shared system, the files can be extracted into a shared location. Users do not need write access to the Launcher program folder. Defaults can be set for users by setting properties in the system-wide properties file.

Launcher Files

Whether or not the Deephaven Installer is used for the installation, the program will be started from a working directory containing the shell script or executable, and supporting JAR and resource files.

Program Files

A Deephaven instance specifies the server and port to which Deephaven should connect for that session. Users can configure multiple instances in the Deephaven Launcher.

The <programfiles>/<instance>/getdown.txt file enables an instance to appear as a choice in the Launcher. For example, if programfiles/foo/getdown.txt exists, then "foo" will be listed as an instance.

By default, program files and log files are stored under the following locations:

Windows	`%LOCALAPPDATA%/illumon`
macOS and Linux	`~/iris/.programfiles`

Workspace Files

Multiple workspaces can be created for each Deephaven instance. In the Launcher, the Workspace drop-down menu will list all available Workspaces for the selected Instance.

By default, workspace files are stored under the following locations:

Windows	`~/Documents/Iris`
macOS and Linux	`~/iris/workspaces`

For example, instance "foo" and workspace "Default" will be in ~/iris/workspaces/foo/workspaces/Default.

Properties Files

When the Launcher starts, it reads properties from all the properties files it can find. System properties always take precedence. Property values will be updated to replace a leading ~ or ${user.home} with the user's home directory.

System-wide properties file:

The system-wide properties file (illumon.launcher.properties) is located in the Launch program folder:

Windows	`%PROGRAMDATA%/illumon/illumon.launcher.properties`
macOS and Linux	`/etc/illumon.launcher.properties`

Per-user properties file:

The per-user properties file (.illumon.launcher.properties) is located in ${user.home}:

~/.illumon.launcher.properties

Launcher properties file:

The Launcher properties file (illumon.launcher.properties) is located in the Launch program folder.

Instance properties file:

This is the .illumon.launcher.properties file located in an instance folder.

Other Properties:

`IrisConfigurationLauncher.instanceRoot`	This overrides the location for the Program files.
`IrisConfigurationLauncher.workspaceRoot`	This overrides the location for the Workspace files.
`IrisConfigurationLauncher.logDir`	The Launcher writes log files in the specified folder, or to `<instance root>/logs` if not specified.
`IrisConfigurationLauncher.logdirsPrevious`	The Launcher and Deephaven programs remove old log files from the log directory. If the log location is changed in `IrisConfigurationLauncher.logDir`, this property can be set to ensure old log files are removed from the previous location.
`IrisConfigurationLauncher.default_instance.url IrisConfigurationLauncher.default_instance.description IrisConfigurationLauncher.default_instance.autocreate IrisConfigurationLauncher.default_instance.autocreate_nickname IrisConfigurationLauncher.default_instance.autocreate_workspace`	These properties can be used to automatically provision initial instances by setting them in the system-wide properties file.
`IrisConsole.logPath`	Governs the log file produced by the console. See: `IrisConfigurationLauncher.logDir`
`IrisConsole.logPaths.previous`	The Launcher and Deephaven programs remove old log files from the log directory. If the log location is changed in `IrisConsole.logPath`, this property can be set to ensure old log files are removed from the previous location.
`IrisConsole.cusAuthString`	If the Client Update Service (CUS) requires authentication, the auth string will be gathered by the Launcher. Because the Deephaven Console checks for updates and needs the same information, it passes the values in this string. Users can set this in a personal properties file to avoid the authentication dialog. See: User Guide to Installing Deephaven.
`IrisConsole.clearPreviousUser`	When this property is set to true, the login screen will clear the previous user from the menu. This defaults to false, which means by default the login screen will remember the previous user.
`Previous_instance Previous_workspace Previous_consolemode`	These properties are used to set defaults in the Launcher UI to the last used settings.
`specified_server_url`	This is set per-instance to the URL used when creating the instance. If there is a problem with the address given in `getdown.txt`, or if the server for instance is changed, this property can be used to redirect the instance.

Last Updated: 16 February 2021 18:07 -04:00 UTC Deephaven v.1.20200928 (See other versions)