The Deephaven Server Installation Guide

This document details the steps to installing Deephaven.

Prerequisites

This section provides information on how to provision a physical or virtual server for Deephaven software.

Deephaven can be installed on a Linux server or virtual machine.

Minimum Hardware Requirements

Processor:	x86_64 (64-bit)
RAM:	128 GB minimum, 512 GB preferred
Database Disk Space:	Mount: /db Size: 100 GB Type: SSD or HDD direct attached.
Intraday Data	Mount: /db/Intraday Size: Sized for several days of the data set. Type: SSD direct attached.
Historical Data	Mount: /db/Systems Size: Depends on the size of the historical data set. Type: Network attached storage (i.e., NFS).
Operating System:	Centos/RHEL version 6.x or greater (ver 7.x recommended) OpenSUSE 12.3 Debian 8.0

Operating System Settings

Turn off swap in the kernel.

echo "vm.swappiness = 0"| sudo tee -a /etc/sysctl.conf
sudo sysctl -p

Note: Many Deephaven processes use large amounts of RAM. If swapping is permitted, the kernel may swap a significant portion of RAM allocated to Deephaven processes. In some environments, this can drastically increase the time needed to run garbage collection (GC). Extremely long GCs can cause network timeouts that adversely impact system stability. For this reason, in addition to general performance concerns, it is strongly recommended that servers running Deephaven are provisioned with ample RAM and configured to avoid swapping.

Set minimum process limits.

Add or adjust the following settings in /etc/security/limits.conf file:

*          soft    nproc     204800
*          hard    nproc     204800
root       soft    nproc     unlimited

Set minimum open files limits.

Add or adjust the following settings in /etc/security/limits.conf file:

cat /etc/security/limits.d/*-nofile.conf
*          soft    nofile     65535
*          hard    nofile     65535
root       soft    nofile     unlimited

Kernel Settings

The use of Transparent Huge Page memory should be disabled.

To check for use of Transparent Huge Pages:

cat /sys/kernel/mm/transparent_hugepage/enabled
always madvise [never]

To disable Transparent Huge Pages in the kernel for Centos 7:

sudo vi /etc/default/grub

Locate the line starting with: GRUB_CMDLINE_LINUX=

Add the following option to the current list: 'transparent_hugepage=never'

sudo grub2-mkconfig -o /boot/grub2/grub.cfg

The IO scheduler should be set to use the 'deadline' scheduler.

Centos 7:

sudo vi /etc/default/grub
Locate the line starting with: GRUB_CMDLINE_LINUX=
Add the following option to the current list: 'elevator=deadline'
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Reboot the server for the settings to take effect.

Install OS Package Dependencies

The following software packages are required for the basic installation. The software's version (when noted) is confirmed to function with Deephaven. However, later versions should be compatible as well.

epel-release
redhat-lsb-core
initscripts
python-pip
monit
wget
unzip
mariadb-server
mariadb
nfs-utils
glibc
libgcc
libgomp
libstdc++
httpd-tools
lighttpd
Python version 2.7 (version 3 is not supported at this time)

sudo yum -y install epel-release

sudo yum -y install redhat-lsb-core initscripts python-pip monit wget unzip mariadb-server mariadb nfs-utils glibc libgcc libgomp libstdc++ httpd-tools lighttpd

sudo systemctl enable mariadb

sudo systemctl start mariadb

Download and Install the Oracle Java JDK *rpm

Download the latest Oracle Java JDK 8 rpm: http://www.oracle.com/technetwork/java/javase/downloads/index.html

Install the downloaded rpm:

sudo yum localinstall -y jdk<VERSION>.rpm*

Install Deephaven Server Software

There are two packages to install: the Deephaven Database package and Deephaven Configuration package. Your Deephaven account representative will provide you with the latest versions of these two packages.

To install the Deephaven server software, you will first need to copy the packages onto your provisioned Deephaven Linux host. Once the packages have been copied to the host, you should SSH onto the server and run the following commands to install the Deephaven Database and the Deephaven Configuration packages:

sudo yum localinstall illumon-db-<VERSION>.rpm -y

sudo yum localinstall iris-config-<VERSION>.rpm -y

The installation includes a default set of configuration files and sample data for a basic Deephaven installation. Note: See Customizing the Deephaven Configuration for more information about customizing your Deephaven configuration.

Install MySql Java Connector Software

Deephaven uses MySql (mariadb) to store authentication and database ACL information. This requires the mysql-connector-java JAR to be installed into /etc/sysconfig/illumon.d/java_lib.

sudo curl -SL http://central.maven.org/maven2/mysql/mysql-connector-java/5.1.43/mysql-connector-java-5.1.43.jar -o mysql-connector-java-5.1.43-bin.jar
sudo cp mysql-connector-java-5.1.43-bin.jar /etc/sysconfig/illumon.d/java_lib/

Modify the Deephaven Configuration

The default Deephaven configuration uses 'localhost'. To configure the various processes to allow connections from a remote console update the following Java properties from 'localhost' to the server FQDN or IP Address:

echo 'FQDNorIP' | sudo tee -a /etc/sysconfig/illumon.d/resources/query_servers.txt

sudo sed -i 's/localhost/FQDNorIP/g' /etc/sysconfig/illumon.d/resources/*.prop

Configure the Client Update Service

The Deephaven Launcher uses the Client Update Service to access code and configuration settings from your Deephaven server. However, before installing and running the Launcher, the Client Update Service must be configured and enabled on the Deephaven server using monit. Follow these steps to enable the Client Update Service:

On the Deephaven Server, edit the /var/www/lighttpd/iris/iris/getdown.txt.pre file. Set the appbase value, replacing WEBHOST with the FQDN or host address of your Deephaven server. The following example demonstrates using 10.20.30.40 as the host address:

#appbase = http://WEBHOST/iris/
appbase = http://10.20.30.40/iris/
#ui.install_error = http://WEBHOST/iris/error.html
ui.install_error = http://10.20.30.40/iris/error.html

In the Deephaven monit configuration folder, remove the .disabled extension from the Client Update Service file name.

cd /etc/sysconfig/illumon.d/monit
sudo mv cus.disabled cus.conf

Start Deephaven Processes

Deephaven processes are started and stopped with Monit. (Refer to https://mmonit.com/monit for more information.)

All Monit configuration files for the Deephaven processes are located in /etc/sysconfig/illumon.d/monit.

Start the monit daemon:

sudo service monit restart

Start the Deephaven processes with the following:

sudo monit start all

This will start all the Deephaven processes enabled in /etc/sysconfig/illumon.d/monit

It may take a few minutes for all processes to start. You can check the status of all processes with the following:

sudo monit status

You can stop all the configured Deephaven processes with the following:

sudo monit stop all

Alternatively, individual Deephaven processes can be stopped and started with the following:

sudo monit stop <process name>

sudo monit start <process name>

You may start individual Deephaven processes with the following:

sudo monit start authentication_server

sudo monit start db_acl_write_server

sudo monit start db_dis

sudo monit start db_ltds

sudo monit start db_query_server

sudo monit start db_ruts

sudo monit start iris_controller

sudo monit start tailer1

You can check the status of individual processes with the following:

sudo monit status <process name>

Access the Client Update Service

Once the Client Update Service is up and running, you can proceed to install and run the Launcher on client desktops. The installers for Windows, Mac and Linux desktops can be downloaded from the Client Update Service on your Deephaven server using any browser:
http://<IRIS_SERVER_ADDRESS>

Default username: iris
Default password: Illumon1
Download the installer for your OS

Verify Your Installation

Start your launcher and connect to the server.
Click New to create a new Instance and enter:
- The URL to your server: http://<IRIS_SERVER_ADDRESS>/iris
- An Instance Nickname
- A Workspace Name
Click OK or Press Enter on your keyboard.
On the next screen, click Launch.
In the next window, enter the User Name and Password when prompted by the site "Client Update Service".

User Name: iris
Password: Illumon1

Enter the default administrative account credentials as shown below:
- Name: iris
- Password: iris
Click Create Console
Select the server in the Query Server drop-down menu, and click Connect
Paste the following query into the console to verify a query against the installed dataset:
t=db.t("LearnIris", "StockTrades").where("Date=`2017-08-25`");
The lower panel should populate with the LearnIris/StockTrades sample data set, as shown below:

Last Updated: 23 September 2019 12:04 -04:00 UTC Deephaven v.1.20181212 (See other versions)