Permissions

We are in the process of updating the information contained in this document. If you need assistance with the steps/processes described within, please contact Deephaven Support.

Overview

Deephaven enables fine-grained access control to data, queries and reports. This is accomplished through the individual authorization of users and groups, access control lists (ACLs) and account information.

Authorized users can edit the access settings and permissions by using the DB ACL Editor tool in the console, or through the command line tool, IrisDbUserMod. The following command will run the tool:

sudo service iris irisdbuser_mod --interactive

Note: Adding a new user to IrisDbUserMod or the ACL Editor does not add the new user to the Deephaven installation. The user must also be permitted to login to Deephaven, which is accomplished during the installation process.

ACL Editor

The ACL Editor can be accessed by clicking the Advanced button in the Deephaven Console.

Note: If you are not authorized by your enterprise to review or modify these settings, the ACL Editor option in the drop-down menu will not appear.

When selected, a dialog window will provide an option to rename the panel, as shown below.

You may rename the panel title or accept the defaults. Click OK to open the DB ACL Editor window, as shown below.

The following tabs are presented at the top of the resulting window:

User/Groups
Accounts (if configured)
Table ACLs
System ACLs
Input Tables

User/Groups

The User/Groups panel shows the available options to manage individual users and groups.

User Administration

The User Administration section (on the left side of panel) includes options for adding and deleting users, or changing an individual user's password, as shown below.

Adding Users

To add a new user, type the new user's username in the field directly under "User Administration". If Managed User Authentication is enabled, there will also be a field for the user's initial password. Click Add User to create the new user.

Note: Adding a new user in this panel only sets the permission/authorization level for a user. It does not add the user to the overall Deephaven application. Also, once a new user is added, the new user can change his/her password at anytime using the Change Password option after clicking the Advanced button in the Deephaven Console.

Deleting Users

To delete a user, select the user's name in the drop-down menu to the left of the Delete User button, then click Delete User. This will delete the user, the user's eponymous group, and all Table ACL entries associated with that user.

Changing Passwords

If Managed User Authentication is enabled, it is possible to change a user's password in the ACL editor. To change an existing user's password, select the user's name in the drop-down menu to the left of the Change Password button, enter the new password in the next field, and then click Change Password.
Note: Change Password operations are only available when iris.enableManagedUserAuthentication is set to true. Additionally, the authentication server must have MySQL authentication enabled via these properties:

authentication.server.customauth.enabled=true

authentication.server.customauth.class=com.illumon.iris.db.v2.permissions.MysqlDbAclProvider

Group Administration

The Group Administration section (on the right side of panel) includes options for adding users to groups, and deleting groups, as shown below.

To add a user to a specific group, select the user's name in the drop-down menu directly below Group Administration. Then select the Group Name in the next drop-down field and click Add User to Group. Note: All new users automatically become a member of the allusers group. Each new user is also added to a group titled with their own username. However, the administration of a group named for an individual user cannot be processed in this panel.
To delete a Group, select the group name from the drop-down list to the left of the Delete Group button, then click Delete Group.

Creating a New Group

Each Group must have at least one member. Therefore, to create a new group, you must first select a user's name in the drop-down menu directly below the Group Administration title. Then, highlight all the text in the Group field to the right, and type in the name of the new group. Click Add User To Group.

The image below demonstrates a user named Fred being added to a new group called NewCoolGroup:

A list showing User names and Group names is presented in the lower portion of the panel.

This list can be filtered by entering an applicable term in the Filter field above the list. Regular expressions can be used in the filter after selecting the Regex check box. Default group mappings can be shown by selecting the Show Default Group Mappings check box.

Deleting a User from a Group

To delete a user from a specific group, find the row in the table that shows the appropriate user's name and the Group name for which he/she should be removed. Right-click anywhere in the row and select Delete User To Group Mappings from the context menu, as shown below.

After Delete User to Group Mappings is selected, a dialog window will prompt you to confirm the deletion. Click Yes to delete the user from the group. Click No to cancel the deletion process.

Special Groups

The following groups are used to provide specific access in Deephaven:

Group Name	Access
`allusers`	All Deephaven users are assigned to this group. By default, members of this group may open interactive consoles and create persistent queries; this is considered a "full-access" group. The name of the group allowed to perform these operations can be changed by updating the following property: `deephaven.access.fullaccess.group=allusers` For example, to create a `deephaven-fullaccess` group for full access, the following property could be added. A user not assigned to the `deephaven-fullaccess` group would not be allowed to create persistent queries or start an interactive console: `deephaven.access.fullaccess.group=deephaven-fullaccess`
`deephaven-noninteractive`	Members of this group are not able to open a console, but will be allowed access through the controller if they have the required privileges (i.e., they can start and stop workers, create queries, and so on). They will also be allowed to connect to a query if they have the required privileges for that query. The name of the group assigned to this privilege level can be changed by editing the following property: `deephaven.access.noninteractive.group=deephaven-noninteractive` This group does not exist when the system is first installed. See: Creating a New Group above.
`deephaven-queryviewonly`	Members of this group can view queries, but are not allowed to create or edit queries. If a user is not assigned to a different privileged group, they are presumed to be in this group (in a default installation, all users are able to create and edit queries because of the `allusers` group). The name of the group assigned to this privilege level can be changed by editing the following property: `deephaven.access.queryviewonly.group=deephaven-queryviewonly` This group does not exist when the system is first installed. See: Creating a New Group above.
iris-acleditors	Members of this group can use the ACL Editor panel.
iris-datamanagers	Members of this group are privileged users who can create import, merge, and data validation queries, and in-worker services. They can also create live queries and batch queries to run on merge servers. Note that a data validation query can delete intraday data, so a member of this group can also do this.
iris-dataimporters	Members of this group can create import queries and in-worker services. They can also create live queries and batch queries on merge servers. If they are not a member of a group with those privileges, they can not create merge or data validation queries.
iris-datamergers	Members of this group can create merge queries.
iris-querymanagers	Members of this group can stop and start all of the queries, but are not allowed to edit them or see the tables (unless otherwise permissioned).
iris-schemamanagers	Members of this group can use the Schema Editor panel.
iris-superusers	Members of this group can view, edit, start, stop, and delete any persistent queries and all tables within a persistent query.

Group Supervisors

Another level of authorization is available for group supervisors. Group supervisors automatically have owner/admin rights for persistent queries owned by anyone in the group. This is accomplished by adding the supervisor to a new group named with the following syntax: <group name>-supervisors.

For example, for a group named quantpros, a supervisor of that group can be granted owner/admin authorization for that group by adding him/her to a new group quantpros-supervisors. Multiple individuals can have the supervisor authorization for a group.

Accounts

The Accounts tab is only visible if configured by the Deephaven system administrator using the DbAclEditor.showAccountTab property.

The Accounts tab only applies to tables filtered with AccountFilterGenerator() and StrategyFilterGenerator(), and provides two levels of mappings from groups to accounts.

Groups are associated with a "Strategy" (e.g., "TradeGroup1", "TradeGroup2" or "TradeGroup3").
"Strategies" are associated with Accounts (e.g., "TradeGroup1" is associated with "ABC_ACCOUNT", "XYZ_ACCOUNT", and "MNO_ACCOUNT").

This extra level of indirection makes it simpler to add a user to a set of accounts.

Table ACLs

The Table ACLs panel enables the authorization of group access to namespaces and tables. Before returning a table to a user, Deephaven applies all of the ACLs for that user.

Row ACLs

Row ACLs determine the rows that a user can see in the table; they cannot change the columns that a user can see in a table. Row ACLs are implemented with where operations that are automatically applied to the table.

Each Row ACL consists of four components:

Group: The name of the group to which this ACL is associated.
Namespace: The namespace associated with this ACL, to apply to all tables. This may be the wild card "*", which means all namespaces are included. Note: If the namespace is "*", the table must also be "*".
Table: The table with which this ACL is associated. The wild card, "*", applies this entry to all tables within the namespace.
Table Filter: A string that represents Java code to create a filter for this table.

Table Filters

Using Java code to generate the filters provides additional flexibility. For example, a filter generator could be new AccountFilterGenerator(), which would filter the Account column based on the accounts for which the current user is authorized.

However, writing Java code to generate the filters can be a bit tedious, so there are two shorthand notations available:

* means that all rows in the table should be presented to the user, and
if a filter generator is of the form whereClause(A, B), whereClause(C, D), then the rows that match (A && B) || (C && D) are available to the user.

To compute the ACLs that should be used for a given user, the following steps are taken:

All of the user's groups are collected into a list.
For each group, get the ACL generator for the table, namespace, or default (in that order).
If no ACL generator was found, then deny permission.
Run all of the ACL generators (unless one returns the empty set of filters, which means permit everything).
If all generators returned null, deny permission (Note: If a generator returns null, that means deny; if it returns an empty list, that means permit everything).
Apply a .whereOneOf() operation to the table with all the returned filters.

When computing the list of ACLs for a table, the most specific ACL for a particular group is used. For example, if "UserXYZ" belongs to the groups "groupABC", "groupXYZ", and "allusers", and if the following ACLs exist:

Group	Namespace	Table	Filter
allusers	*	*	new OwnNamespaceGenerator()
groupABC	SystemEQ	PositionCache	new AccountFilterGenerator()
groupABC	SystemEQ	*	*
groupXYZ	SystemEQ	*	whereClause(`false`)

Then, the ACL generators that would be used for the SystemEQ.PositionCache table are:

new OwnNamespaceGenerator() for allusers, because no more specific ACL exists for that group.
new AccountFilterGenerator() for groupABC, because SystemEQ.PositionCache is more specific than SystemEQ.* (the "*" filter would be unused in this case)
whereClause(`false`) for groupXYZ

The OwnNamespaceGenerator would return null because the SystemEQ namespace is not equal to the username UserXYZ, thus contributing no filters. The AccountFilterGenerator would return a match filter for the account column. Finally, the whereClause(`false`) produces a match filter with no results. This has the effect of allowing the user to see an empty table, but without denying permission. When taken together, groupABC could see rows in SystemEQ.PositionCache that have the appropriate accounts configured.

If any other table from the SystemEQ namespace were requested, then the SystemEQ.* ACL would be the most specific ACL for group ABC, resulting in an ACL of "*", which would short-circuit further evaluation; and all rows would be visible.

Note: Adding a user to a group can only increase the permissions enabled for that user. Users cannot lose any existing permissions by being added to another group (e.g., if a group has the NullFilterGenerator() applied, any pre-existing user permissions would not be impacted.)

Column ACLs

Column ACLs provide administrators a way to restrict access to data on a columnar level. This stands in contrast to Table ACLs which restrict access by explicitly filtering rows out of the table. This can be useful when all users must have access to some subset of the data in all rows of a particular table. For example, a manager may need to be able to monitor the current execution of all orders within a system, but may not be allowed access to pricing data for some, or all of the symbols.

The rules for Column ACL application follow the same rules as for Table (Row) ACLs. The most specific ACL for each group is the one applied. Additionally, Column ACLs are defined with a comma-separated list of column names to which the ACL applies.

Each Column ACL consists of five components:

Group: The name of the group to which this ACL is associated.
Namespace: The namespace associated with this ACL. Use an asterisk ( * ) to apply to all namespaces.
Table: The table with which this ACL is associated, or * for all tables.
Applicable Columns: The set of columns in the table for which this ACL applies, or * for all columns. Note that the columns must exist for the table or tables denoted by the above parameters.
Column Filter: A string that represents the filter for the specified columns.

Filter Generators

The filter generator string in the ACL should create a FilterGenerator object. Most of these strings will use one of the following built-in filter generators:

`EmptyFilterGenerator()`	Provides full access to a table; it can be abbreviated as "`*`".
SimpleFilterGenerator()	Passes through a simple where clause; it can be abbreviated by using a filter generator string beginning with `whereClause`.
AccountFilterGenerator()	Creates a match filter for accounts for which this user is authorized; suitable for restricting position logs, trade logs, etc. to just the accounts a user should see. By default, this filters the Account column, but an optional column name can be passed into the generator.
StrategyFilterGenerator()	Creates a match filter for strategies for which this user is authorized. By default, this filters the Strategy column, but an optional column name can be passed in to the generator.
OwnNamespaceFilterGenerator()	Produces an empty filter (i.e., allow everything) when the namespace matches the effective user name; otherwise produces null (i.e., no filter). This allows users to have full access to their own user tables.
UsernameFilterGenerator()	Creates a match filter for this user's username. By default, this filters the Username column, but an optional column name can be passed in to the generator.
UserCollectionFilterGenerator	Creates a filter for this user's username within a collection. By default, this filters the Username column, but an optional column name can be passed in to the generator. This is useful, for example if a table column contains a list of authorized users for each row.
GroupFilterGenerator()	Creates a match filter for groups to which this user belongs. By default, this filters the Group column, but an optional column name can be passed in to the generator.
WorkerNameFilterGenerator()	Produces a filter that matches only this worker. By default, this filters the `WorkerName` column, but an optional column name can be passed in to the generator. This is useful for filtering query logs. Note: the `QueryPerformanceLog`, `UpdatePerformanceLog` and `QueryOperationPerformanceLog` contain several user name columns which may be used for filtering in the latest version of Deephaven; so this `FilterGenerator` is no longer commonly used.
RequestIdFromWorkerNameFilterGenerator()	Produces a dynamic where filter based on the `DbInternal.QueryPerformanceLog` table (we know the worker of a database, and this log lets us correlate a worker with a request ID). By default, this filters the `RequestId` column, but an optional column name can be passed in to the generator. Note: the `QueryPerformanceLog`, `UpdatePerformanceLog` and `QueryOperationPerformanceLog` contain several user name columns which may be used for filtering in the latest version of Deephaven; so this `FilterGenerator` is no longer commonly used.
NullFilterGenerator()	Provides no access to a table; only useful to override a less-specific ACL for a group.
ConjunctiveFilterGenerator(fg1, fg2, ...)	Runs filter generators `fg1`, `fg2`; and returns a filter that is the conjunction of the filters returned by the passed-in filter generators. You can use this to further restrict the output of another filter generator.
CopyFilterGenerator(Namespace, Table)	Copies the ACLs from another table to this table.
CombiningFilterGenerator(Type)	ACLs are normally applied disjunctively. In some cases, users may want to override this behavior and combine some group filters conjunctively instead. This generator is initialized with the operation type desired (Type.Conjunctive or Type.Disjunctive) and as a single method addFilter(group, generator) to which specific filter generators can be added. Note that this generator is intended for programmatic ACLs.

Permission Analyzer

The Permission Analyzer can be used to review the table permissions associated with any particular user. To access this tool, click the Permission Analyzer button on the Table ACLs panel. Then select the appropriate User, Namespace and TableName in the drop-down menus to review the permissions granted. The example below shows the permissions for user123 in regard to the DXFeed namespace and the TradeStock table. The bottom part of the panel shows that full read permission is allowed for that user (as noted by the *).

Selecting the Show groups without ACLs check box adds the names of the groups associated with that user and each group's respective permission levels, as shown below.

System ACLs

The System ACLs panel permits users in the acleditors group to add, remove, and edit system ACLs, similar to how they can add, remove, and edit table ACLs.

Each System ACL consists of three components:

Group: The name of the group to which this ACL is associated.
Key: System ACLs have a single key and are are not limited in scope to a single namespace or table. This key may be entered directed in the Key field and does not need to be predefined.
Filter: A string that represents Java code to create a filter. This works the same way as Table Filters.

Input Tables

Setting the authorization for individual/group access for reading data in tables is accomplished through the Table ACLs tab. However, setting the authorization of individuals or groups to edit tables is accomplished in the Input Tables tab.

Note: When a user or group is authorized to edit a table, the entire table may be edited. Setting authorization for editing only a portion of the table is not supported.

Persistent Query ACLs

Just as Access Control Lists can be applied to tables, they can also be applied to persistent queries. The owner of a query, as well as designated administrator groups, have full access to the query. They can see every table and are able to edit the query, restart it, stop it, etc. The owner and administrator can also select groups that can view the query. Viewer groups are not able to view the query source code, stop, start, or edit the query.

However, permissions do not carryover to the plots generated by those same persistent queries. To set access control for viewing these plots, you must use the setValidGroups method in the persistent query that generates the plot. See below for more about Setting Permissions for Viewing Plots.

Tables

As part of the Groovy code, filters can be added to tables. These filters do not take effect for the owner, admin, or members of the iris-superusers group; only for viewers. If no filters are defined on any table, then viewers can see all of the rows in all of the tables produced by the query. If there is a filter defined for at least one table, then viewers cannot see any tables without filters. This has the effect of making it convenient to let people see the full results of a query. Assuming care is taken to restrict part of the query, the parts that have not been addressed from a security perspective are hidden.

Row and Column filters are applied to tables using a TableFIlterProvider instance, created as follows:

TableFilterProvider.FACTORY.create(db, table)

Operations to add Row or Column ACLs can then be chained to apply the desired permissions to the table.

Row ACLs are applied using the following method:

.addFilter("group", "filter1", "filter2", ...)

Where db is the database object, group is the group to which access is provided, and filter1, filter2, etc. are filter generator expressions to add (disjunctively). The format of the filter expressions is the same as for Table ACLs defined in the DB ACL Editor.

Column ACLs are applied using the following method:

addColumnACL(“group”,”Column1,Column2,...”, “filter1”, “filter2”, ...)

Filters can be chained, for example:

TableFilterProvider.FACTORY.create(db, table)
.addFilter("group1", "*")
.addFilter("group2", "whereClause(\"A=`B`\"")
.addFilter("group2", "whereClause(\"A=`C`\", \"D=`E`\")")
.addColumnACL(“group3”, “Bid,Ask”, “new GroupFilterGenerator(\”MarketDataGrp\”)”)

In this example, group1 may access the entire table, but group2 can only view rows where A is equal to B, or rows where A is equal to C, and D is equal to E. Additionally, group3 can only view the specified rows.

Setting Permissions for Viewing Plots

The ability to see and edit persistent queries and the tables they generate is determined by the settings used by the query author in the Access Control tab of the Persistent Query Configuration panel. However, those permissions do not carryover to the plots generated by those same persistent queries.

To set access control for viewing these plots, you must use the setValidGroups method in the persistent query that generates the plot.

Syntax

setValidGroups(String... validGroups)

setValidGroups(Collection<String> validGroups)

To limit viewing access to a particular plot, first write the query to create and show the plot. Then, assign the setValidGroups method to that plot. Note: in a Python query, the .setValidGroups argument must be on a separate line. Arguments to setValidGroups method include the names of the User(s) and Group(s) you want to be allowed to see the plot.

For example, the following query creates an XY Series chart to show the price of a single security (PFE) over the course of one day. No viewing restrictions are currently in place.

t1 = db.t("LearnDeephaven","StockTrades").where("Date=`2017-08-24`","USym=`PFE`")
PlotPFE = plot("PFE", t1, "Timestamp", "Last")
.xBusinessTime()
.show()

To set the viewing permissions for this plot, add the following line to the query.

PlotPFE.setValidGroups("User1", "User2", "Group1")

When this query is executed in Deephaven, only User1, User2 and individuals included in Group1 would be allowed to open and view the plot using the Show Widget button in the Deephaven console.

For example, if User3 was not included in Group1, User3 would not be able to view PlotPFE even if User3 was authorized to view the persistent query and/or the tables generated by the persistent query.

Note: The setValidGroups method will have the same effect as the addFilter method (described above in the Tables section) in that, if used, any table without restrictions will also be hidden.

Keep in mind that the setValidGroups method does not differentiate the data shown to each user. The widget can make use of the IrisWidgetSupport getIrisUserContext method to restrict information based on the user.

The LiveWidgetVisibilityProvider interface can be implemented by a widget to restrict the set of users that can load the widget. However, it does not differentiate the data shown to each user. The widget can make use of the IrisWidgetSupport getIrisUserContext method to restrict information based on the user.

Authentication Server Third-Party ACL Integration

System administrators are able to create and implement Deephaven Authentication Server plug-ins that will execute Java code upon certain Authentication Server actions, such as user login or token verification.

This document describes a specific use case in which an "Auth Hook" (plug-in) can be created to request and parse data from an external server and then import that data into the Deephaven system. The imported data may then be utilized in the ACL Editor by a corresponding, custom filter generator.

The following graphic demonstrates the general process:

Setup

There are four steps required to implementing the above system:

Create the authentication server plug-in by creating a class that implements the AuthHookModule interface.
Specify the plug-in through the authentication.server.hooks.class property.
Set up a tailer to import logged data real-time.
Create a custom filter generator that utilizes the imported data within an ACL.

AuthHookModule Interface

To create your authentication hook, you must create a class that implements the com.fishlib.auth.AuthHookModule interface and overrides its methods.

The AuthHookModule interface follows:

package com.fishlib.auth;
/**
 * This interface allows you to run actions after useful authentication server operations; for example you may use
 * an AuthHookModule to perform post-login refresh of ACL entries.
 *
 * Implementations must implement a Constructor which takes a com.fishlib.io.logger.Logger.
*/
public interface AuthHookModule {
    /**
     * Called after a client successfully logs in to the authentication server.
     *
     * userContext is the context of the user logging in
     *
    void onLogin(UserContext userContext)
				
    /**
     * Called after a client logs out from the authentication server (i.e. disconnects).
     *
     * userContext is the context of the user logging out
     */
    void onLogout(UserContext userContext);
				
    /**
     * Called after an authentication token is created.
     *
     * authenticatedContext is the context of the user creating the token
     * service is the service the user is creating a token for
     */
    void onTokenCreate(UserContext authenticatedContext, String service);
    /**
     * Called after an authentication token is verified
     *
     * authenticatedContext is the context of the verified token
     * service is the service for which the token is valid
     */
    void onTokenVerify(UserContext authenticatedContext, String service);
				
    /**
     * Called after an authentication token fails verification
     *
     * authenticatedContext is the context of the verified token
     * service is the service for which the token is valid
     */
    void onTokenVerifyFailure(UserContext authenticatedContext, String service);
     /**
     * Called after the authentication server configuration is reloaded.
     */
    void onReload();
				
  class Null implements AuthHookModule {
				
        @Override
       public void onLogin(UserContext userContext) {
				
        }
				
        @Override
        public void onLogout(UserContext userContext) {
				
        }
				
        @Override
        public void onTokenCreate(UserContext authenticatedContext, String service) {
				
        }
				
        @Override
        public void onTokenVerify(UserContext authenticatedContext, String service) {
				
        }
				
        @Override
        public void onTokenVerifyFailure(UserContext authenticatedContext, String service) {
				
        }
				
        @Override
        public void onReload() {
				
        }
    }
}

The nested com.fishlib.auth.AuthHookModule.Null class provides an implementation with empty methods, which you may use as your base class. By using the Null implementation as your base, you only need to implement methods required for your use case, and your extension will continue to compile if additional methods are added to the AuthHookModule interface.

Authentication Server Hooks Property

To utilize the AuthHookModule implementation, you must specify the fully qualified class name of the hook in the authentication.server.hooks.class property:

authentication.server.hooks.class=<fully qualified class name>

Data Import

Authentication hooks may utilize a logger like a client logger application would. If this is the case, you will need to set up a tailer to import the logged data real-time. Please refer to Importing Data > Deephaven Data Tailer for an example of tailer installation and configuration.

Custom Filter Generator

If you'd like to utilize the imported data in the ACL Editor, you must create a class that implements the FilterGenerator interface, and then apply it as an ACL. Refer to the Deephaven Javadocs and the ACL Editor documentation to learn more.

DACS Integration

The DACS Integration plug-in provides a standard method to apply Thompson Reuters DACS entitlements to data provided by the Deephaven System. Once installed and applied to a table, the system will automatically filter it down to the set of items an individual user has permission to view, as well as record usages per user to be used for compliance auditing.

The plug-in is composed of three main components:

An Authentication Hook, which allows the system to query DACS for an authenticated user's set of permissioned entities.
A Filter Generator used to apply DACS permissions on demand to tables requested by users.
A set of three system tables under the DACS namespace to record entity mapping, user permissions and per-user entity usage.

Upon login, the plug-in will request a particular user's set of permissioned entities (PEs) and record them to a system table Dacs.PermissionMatrix. Once this is complete, and until the user completely logs out of the Deephaven system, this list of PE's will be dynamically maintained from DACS.

After this first step is completed, if the user requests a table that has been connected to a DACS Filter Generator, the plug-in will automatically apply a filter to the requested table that will only pass rows for which the user has proper DACS entitlements for. If the user's set of entitlements is changed in DACS, then that table will be re-filtered with the updated permission set, allowing for intraday permission change compliance.

In addition to only passing rows for which a user has entitlements, the plug-in will record item usage per user to an additional system table DACS.Usage. This table can then be used to generate daily per-user usage reports to be used for compliance auditing.

Installation

In order to function properly the plug-in must be installed on each host that is part of a Deephaven cluster. Installation may be performed from one of two package options, RPM and TAR archive.

RPM Installation (Preferred)

Installing the integration from an RPM is the simplest method. Place the RPM on the host it needs to be installed on and run the following command:

sudo yum install dacs-integration-<version>.x86_64.rpm

TAR Installation

The tarball installation supports systems where security policies do not allow for the yum package manager to be run as sudo, or are otherwise too restrictive to permit installation from an RPM. Note that all of the commands below should be executed as irisadmin (not root). It does not matter if this is done via prefixing them with sudo -u irisadmin or directly as that user.

First, copy the tar to the server using your preferred method (e.g., rsync, scp, or sftp). For example:
rsync -avz --rsync-path='$([ -d /tmp/dacs-integration/ ] || mkdir -p /tmp/dacs-integration/) && rsync' -e "ssh -A -i [~/.ssh/your_key_here]" /dh/ws1/dacs-integration/build/distributions/Dacs-integration-[version]-Manual.tgz [your_IP_address]:/tmp/dacs-integration/
Then ssh into your remote machine:
ssh -A -i ~/[.ssh/your_key_here] [your_IP_address]
Next, prepare the installation directory:
rm -rf /etc/sysconfig/illumon.d/plugins/dacs-integration
mkdir -p /etc/sysconfig/illumon.d/plugins/dacs-integration
Then, extract the tar into that directory:
tar -xzf dacs-integration-<version>-Manual.tar -C /etc/sysconfig/illumon.d/plugins/dacs-integration
Ensure that Deephaven is properly configured to pull schemas from plug-ins. Depending on your installation version this can be in one of two files:
- /etc/sysconfig/illumon.d/resources/IRIS-CONFIG.prop
- /etc/sysconfig/illumon.d/resources/iris-common.prop (Deephaven v1.20190322 or later)
Look for the property SchemaConfig.resourcePath.plugins. If this is present, you do not need to take further action. If it is missing, add the following to the file:
SchemaConfig.resourcePath.plugins=/etc/sysconfig/illumon.d/plugins/*/schema/
Next, run the installation configuration scripts:
/etc/sysconfig/illumon.d/plugins/dacs-integration/bin/reinstall.sh

Configuration

There are only a few configuration options required for the integration to function. Note that there are two possible ways properties are defined, depending on your Deephaven Installation.

Older installations may use the "split and include" pattern of configuration. In this variant, there will be several .prop files in /etc/sysconfig/illumon.d/resources. We are only concerned with the following:

/etc/sysconfig/illumon.d/resources/IRIS-CONFIG.prop /etc/sysconfig/illumon.d/resources/iris-authentication-server.prop

Newer installations use the common properties file, where there is a single file -

/etc/sysconfig/illumon.d/resources/iris-common.prop - which is internally separated into sections that are relevant for each configuration.

First, the plug-in must be configured as an Authentication Hook. These should be added to the authentication server properties as described above.
If the current Deephaven Installation is not already configured with an auth hook, set the following property:
authentication.server.hooks.class=io.deephaven.dacs.auth.DACSAuthHook
If the installation is already configured with an auth hook, it must be reconfigured as a compound auth hook as follows:
authentication.server.hooks.class=io.deephaven.dacs.auth.CompoundAuthHook
dacs.auth.hooks=io.deephaven.dacs.auth.DACSAuthHook,<Other Auth hook>
Next, the DACS configuration properties must be set in the global properties:
dacs.host=<Your Dacs Host>:<Dacs Port>
dacs.service=<The DACS Service name to use for permissioning>
dacs.applicationId=<The ApplicationID assigned to Deephaven by DACS>
dacs.ignoreUsers=iris,root,superuser
After Steps 1-2 are completed, you must restart the authentication server to load the updated properties:
sudo monit restart authentication_server
After Step 3, the plug-in will begin to record user permission lists upon login. These can be inspected via query using the following command:
db.i("DACS","PermissionMatrix").where("Date=currentDateNy()")

At this point you may add DACS Filter generators to tables via ACLs or directly in queries.

Tailoring

The DACS system derives entitlements from what is called a DACS Lock. This is a potentially complicated combination of (Service Id, PE List, Condition) tuples that can not be pulled directly from the DACS System. Instead these DACS Lock objects are provided by the Thomson Reuters streaming APIs when subscriptions are made. Because of this API and registration pattern, the integration plug-in must be provided with a mapping of items to be permissioned to their opaque DACS Lock bytes. For this purpose, the integration includes a table schema called DACS.ItemMap.

This schema consists of three columns:

Timestamp - a Long column representing an instance in nanoseconds
Item - A String representing the item to associate with the DACS Lock (i.e., RIC)
LockData - A blob (byte[]) of the DACS Lock bytes provided by the Subscription

The Integration includes a generated logger class io.deephaven.dacs.gen.ItemMapFormat1Logger that may be used to write this table from a Java application. C# and C++ loggers may also be used as long as they conform to the DACS.ItemMap schema.

Using The Provided Java Logger

If desired, the supplied Java logger class may be used to write this table. This class should not be directly instantiated. It should instead be created using the built in logger factory facilities as follows:

import io.deephaven.dacs.gen.ItemMapFormat1Logger;
import com.illumon.iris.db.util.logging.EventLoggerFactory;

public ItemMapFormat1Logger createLogger(Logger log) {
	final Configuration config = Configuration.getInstance();
	return EventLoggerFactory.createIntradayLogger(config,
		"DACSAuth", 
		log, 
		ItemMapFormat1Logger.class,
		config.getServerTimezone().getID());

}

public void exampleLog(ItemMapFormat1Logger logger, String item, byte[] lockBytes) {
	logger.log(DBDateTime.now().getNanos(), item, lockBytes);
}

Using this option requires no additional configuration. The created binary log files will be written into /var/log/deephaven/binlogs and will be automatically picked up by the default tailer instance.

Logging From C++ or C#

The included DACS.ItemMap schema includes a definition for a C# logger which can be generated via generate_loggers_listeners. To write C++ loggers, refer to the C++ logger documentation, matching the provided schema.

These logger types allow users to place the result binary log files anywhere on the filesystem with any desired file name. In order to guarantee that these binary logs are imported by a tailer, an additional tailer configuration must be set up. Note that it is recommended to use a filename of the format: <Namespace>.<Tablename>.Main.bin.<Date>

The integration comes packaged with a set of template tailer configuration files that must be tailored to match where the C++ or C# logger will write its binary log files.

First, edit the template tailer configuration file:
/etc/sysconfig/illumon.d/plugins/dacs-integration/samples/tailerConfigDACS.xml
Change the attribute logDirectory to the directory where the logger will write the binary log files.
Move this file into /etc/sysconfig/illumon.d/resources
Edit the included tailer properties file: /etc/sysconfig/illumon.d/plugins/dacs-integration/global/props/tailerdacs.prop
Change the property log.tailer.enabled.dacs=false to true.
Finally, restart the tailer:
sudo monit restart tailerdacs

Attaching Filters

Once the configuration and tailoring of the integration is complete, filters can be added to tables to apply entitlements and record usages.

Filters as ACLs

One of the simplest ways to apply entitlements is using table ACLs. The integration contains one ACL class io.deephaven.dacs.acl.DACSFilterGenerator. This class requires a single parameter on construction to select the column that contains the items to be filtered by. An example ACL to use in the Table ACL editor would be:

new io.deephaven.dacs.acl.DACSFilterGenerator("USym")

This ACL will filter the table it is applied to by checking entitlements against the USym column.

Programmatic Filters by Query

Table ACLs are appropriate when users are accessing tables from their own queries or a standalone console. When a single query is used to provide data to multiple users, the query writer must apply appropriate ACLs to the tables served by the query to ensure that proper DACS Entitlements are applied to the derived tables.

The following groovy closure can be used as a starting template:

addDACSFilter = { Table t, String filterColumn ->
	def filterProvider = t.getAttribute(Table.ACL_ATTRIBUTE);
	if(filterProvider == null) {
		filterProvider = TableFilterProvider.FACTORY.create(db, t)
	}
	filterProvider.addFilter("allusers", 'new io.deephaven.dacs.acl.DACSFilterGenerator("'+filterColumn+'")')
}

This is applied to tables in the following manner:

addDACSFilter(myDerivedTable, "USym")

Usage and Compliance

Many data providers require usage auditing per user during feed access. The integration plug-in writes out a table called DACS.Usage which contains usages per user by item of all items that were provided by Deephaven. This table has four columns: Date, Timestamp, User, and Usages.

The Usages column is an array of Strings that can be ungrouped to create a list of all visited items. Below is a simple query that will produce a set of plaintext files containing all unique usages per user:

usages = db.i("DACS", "Usage").where("Date=currentDateNy()")
usagesByUser = usages.byExternal("User")
		.asTable()
		.ungroup()
		.selectDistinct("Usages")
		.renameColumns("Usage=Usages")
		.asTableMap()

for(Object k : usagesByUser.getKeySet()) {
	final String userName = (String)k;
	final Table t = usagesByUser.get(userName);

	writeCsv(t, "/db/TempFiles/audit/usages."+userName+"."+currentDateNy()+".csv")
}

Last Updated: 28 February 2020 12:20 -05:00 UTC Deephaven v.1.20200121 (See other versions)