Remote Processing Profiles

Remote processing profiles are used to specify JVM parameters and environment variables for persistent queries and interactive consoles. Several predefined profiles are available, and new ones can be defined using controller properties (see Custom below). Remote processing profiles can be dynamically reloaded by using the controller tool's reload command.

These profiles are detailed below.

Classic (CMS)

The name of this profile is "CMS GC".

The classic profile uses Java's CMS (Concurrent Mark Sweep) garbage collection. It uses a set of garbage collection JVM parameters that are intended to limit the number of CPUs any single Remote Query Processor can consume for garbage collection. These Deephaven default parameters can be overridden by defining the RemoteQueryDispatcher.ClassicJVMParameters property. In addition, if it is defined, the RemoteQueryDispatcher.JVMParameters property will add further parameters when the JVM is started (and will also add those parameters to the Garbage First profile).

Garbage First (G1)

The name of this profile is "G1 GC".

The Garbage First profile uses Java's newer G1 (Garbage First) garbage collection. It uses a default set of parameters, which can be overridden with the RemoteQueryDispatcher.G1JVMParameters property. In addition, if it is defined, the RemoteQueryDispatcher.JVMParameters property will add further parameters when the JVM is started (and will also add those parameters to the Classic profile).

If the RemoteQueryDispatcher.G1NumberHeapRegions property is defined, this profile will define the Java -XX:G1HeapRegionSize parameter based on standard G1 tuning recommendations.

Garbage First (G1) with Custom MarkStackSize

The name of this profile is "G1 MarkStackSize 128M".

For some queries, the default G1 MarkStackSize values may not be large enough. This will be manifested by full garbage collection cycles, indicated with messages like the following in the garbage collection log:

[Full GC (Allocation Failure) ...]

Messages like the following may also be seen in the logs:

[GC concurrent-mark-reset-for-overflow]

In this case, the GC profile may reduce or eliminate the full garbage collection cycles. This profile adds the following parameters to the JVM:

-XX:MarkStackSize=128M

-XX:MarkStackSizeMax=256M

None

The name of this profile is "None".

This profile indicates that no extra parameters are to be sent to the JVM when it is started. It is still possible to define parameters with the persistent query, or at the console start screen.

Default

The name of this profile is "Default".

This indicates that another administrator-defined profile is to be used (the default profile). This can be useful in case an administrator wants to define the default behavior, for example as classic or G1, and be able to change it for all queries using this default profile. It uses the RemoteQueryDispatcher.defaultJVMProfile property to define the name of the profile to be used, and defaults to "CMS GC".

Custom (Defined with Properties)

The names of these profiles are assigned in the properties.

An administrator can define custom profiles by adding properties to the controller's property file. Each custom profile is assigned a name which is used to define the properties. Properties are always in the format:

RemoteProcessingRequestProfile.custom.<profile name>.<parameter type>.<parameter name>=value

<profile name> - the custom profile's name. A custom profile may have several properties, but all of them will start with RemoteProcessingRequestProfile.custom.<profile name>. All properties starting for same profile name are applied to that profile. The name will be displayed to users in the profile drop-down box.
<parameter type> - the parameter type, as detailed below. This indicates something that is being defined for this profile.
<parameter name> - the parameter's name. Parameter names are arbitrary, and are only used to distinguish different parameters of the same type.

Parameter types

The parameter type defines a specific thing that is being applied for a profile. The following parameter types are supported. All examples assume that the profile is called trialProfile.

jvmParameter - one or more JVM arguments to be passed to the JVM when it is started. For example:
RemoteProcessingRequestProfile.custom.trialProfile.jvmParameter.deDup=-XX:+UseStringDeduplication
RemoteProcessingRequestProfile.custom.trialProfile.jvmParameter.newProp=-DmyProp=value
jvmAppendableParameter - JVM parameters which are combined into a single argument passed to the JVM when it is started. All these parameters must be in the form <parameter name>=<parameter value>, and all the values with the same name will be combined into a single JVM parameter using the system's path separator. For example, the following two properties could be defined in different property files both of which are included by the controller:
RemoteProcessingRequestProfile.custom.trialProfile.jvmAppendableParameter.libPath1=-Djava.library.path=/plugins/plugin1/bin
RemoteProcessingRequestProfile.custom.profileName.jvmAppendableParameter.libPath2=-Djava.library.path=/plugins/plugin2/bin
They will be combined into the following single parameter for the JVM:
-Djava.library.path=/plugins/plugin1/bin:/plugins/plugin2/bin
These appendable parameters are included across profiles if one profile includes another . The string used to join them (by default ":" or ";" depending on the operating system) can be changed with the following property (note that this is specific to the property name, not the profile):
RemoteProcessingRequestProfile.joinString.<parameter name>
environmentVariable - environment variables to be defined for the JVM. These are always defined in the format <environment variable name>=<value>. For example:
RemoteProcessingRequestProfile.Custom.trialProfile.environmentVariable.example1=my_env_variable=someValue
include - specifies one or more profiles (by name) to be included in this profile. Multiple profiles can be included either by using a comma-delimited list, or by using multiple .include directives, each with its own name. If a profile is included twice, this is a configuration error and causes an exception. For example:
RemoteProcessingRequestProfile.custom.trialProfile.include=G1 GC,customBaseProfile
RemoteProcessingRequestProfile.custom.trialProfile.include.second=customProfile3

Examples

The following custom profile adds a a listening debugger:

RemoteProcessingRequestProfile.custom.Debug_5005.include.1=Default
RemoteProcessingRequestProfile.custom.Debug_5005.jvmParameter.1=-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005

The following custom profile is for use with JProfiler:

RemoteProcessingRequestProfile.custom.Profile_8849.include.1=Default
RemoteProcessingRequestProfile.custom.Profile_8849.jvmParameter.1=-agentpath:/opt/jprofiler10.1.2/bin/linux-x64/libjprofilerti.so=port=8849,nowait

Additional Properties

The following property is not applied to one profile, but across the system:

RemoteProcessingRequestProfile.defaultProfile - this defines the profile name that is selected for new persistent queries and in the console connection screen (unless another is saved in the user's workspace). Unless a user specifically changes the profile (by checking the "Advanced Options" checkbox and changing the profile), any new persistent query will use this default profile. Changing this property does not change the profile for any existing queries, just for newly-created ones.

Last Updated: 06 July 2020 14:55 -04:00 UTC Deephaven v.1.20200928 (See other versions)