Package com.illumon.iris.db.tables.remotequery

Class RemoteQueryClient

java.lang.Object
com.illumon.iris.db.tables.remotequery.RemoteQueryClient
All Implemented Interfaces:
com.fishlib.dataobjects.HashCommandConstants, RemoteQueryCommandConstants, io.deephaven.enterprise.comm.api.CommandHandler, io.deephaven.enterprise.comm.BaseCommandConstants
public class RemoteQueryClient extends Object implements io.deephaven.enterprise.comm.api.CommandHandler, RemoteQueryCommandConstants
Interface for remotely connecting to Deephaven.

  • Field Details

    • REMOTE_DB_WORKER_REQUEST_TIMEOUT_MILLIS

      public static final long REMOTE_DB_WORKER_REQUEST_TIMEOUT_MILLIS

    • DEFAULT_WORKER_PRIORITY

      public static final int DEFAULT_WORKER_PRIORITY
      The default priority of a worker, for when it's not overridden (for example by the query type). Lower numbers are higher priorities, with 0 as the highest. Negative numbers are not allowed.

  • Constructor Details

    • RemoteQueryClient

      public RemoteQueryClient() throws IOException
      Create a new Deephaven dispatcher connection. The default host and port are selected using the RemoteQueryDispatcherParameters.host and RemoteQueryDispatcherParameters.queryPort properties.
      Throws:
      IOException - if an error occurred connecting to the RemoteQueryDispatcher

    • RemoteQueryClient

      public RemoteQueryClient(@Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams) throws IOException
      Create a new Deephaven dispatcher connection. The default host and port are selected using the RemoteQueryDispatcherParameters.host and RemoteQueryDispatcherParameters.queryPort properties.
      Parameters:
      connectionMonitorParams - heartbeat and idle monitoring parameters
      Throws:
      IOException - if an error occurred connecting to the RemoteQueryDispatcher

    • RemoteQueryClient

      public RemoteQueryClient(String serverHost, int serverPort) throws IOException
      Create a RemoteQueryClient and connect it to the RemoteQueryDispatcher.
      Parameters:
      serverHost - the hostname of the RemoteQueryDispatcher
      serverPort - the port of the RemoteQueryDispatcher
      Throws:
      IOException - if an error occurred connecting to the RemoteQueryDispatcher

    • RemoteQueryClient

      public RemoteQueryClient(String serverHost, int serverPort, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams) throws IOException
      Create a RemoteQueryClient and connect it to the RemoteQueryDispatcher. Uses the default authentication client manager.
      Parameters:
      serverHost - the hostname of the RemoteQueryDispatcher
      serverPort - the port of the RemoteQueryDispatcher
      connectionMonitorParams - heartbeat and idle monitoring parameters
      Throws:
      IOException - if an error occurred connecting to the RemoteQueryDispatcher

    • RemoteQueryClient

      public RemoteQueryClient(String serverHost, int serverPort, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, @NotNull TokenFactoryFactory tokenAuthenticationManager) throws IOException
      Create a RemoteQueryClient and connect it to the RemoteQueryDispatcher.
      Parameters:
      serverHost - the hostname of the RemoteQueryDispatcher
      serverPort - the port of the RemoteQueryDispatcher
      connectionMonitorParams - heartbeat and idle monitoring parameters
      tokenAuthenticationManager - the authentication client manager to use
      Throws:
      IOException - if an error occurred connecting to the RemoteQueryDispatcher

  • Method Details

    • getDefaultClassPushList

      public static List<RemoteQueryClassDefinition> getDefaultClassPushList()

    • getDefaultClassPathList

      public static String[] getDefaultClassPathList()

    • getClassPushList

      public static List<RemoteQueryClassDefinition> getClassPushList(String resourceName)

    • connectToNewProcessor

      public static QueryProcessorConnection connectToNewProcessor(@NotNull com.fishlib.io.logger.Logger log, @NotNull QueryDispatcherConnection dispatcherConnection, @Nullable AuthToken authTokenForDispatcher, @Nullable AuthToken authTokenForProcessor, @NotNull QueryDispatcherConnection.QueryProcessorRequestHandle handle, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, @NotNull TokenFactoryFactory tokenAuthenticationManager, String[] classpath, int priority, int maxHeapSizeMb, String queryDescription, boolean debug, boolean detailedGCLogging, boolean omitDefaultGcParameters, boolean exportQueryScope, List<RemoteQueryClassDefinition> classPushList, Map<String,String> extraEnvironmentVars, String... jvmArgs) throws RemoteQueryException
      Throws:
      RemoteQueryException

    • connectToNewProcessor

      public static QueryProcessorConnection connectToNewProcessor(@NotNull com.fishlib.io.logger.Logger log, @NotNull QueryDispatcherConnection dispatcherConnection, @NotNull QueryDispatcherConnection.QueryProcessorRequestHandle handle, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, @NotNull TokenFactoryFactory tokenAuthenticationManager, RemoteProcessingRequest request, boolean exportQueryScope, List<RemoteQueryClassDefinition> classPushList) throws RemoteQueryException
      Throws:
      RemoteQueryException

    • connectToNewProcessor

      public static QueryProcessorConnection connectToNewProcessor(@NotNull com.fishlib.io.logger.Logger log, @NotNull QueryDispatcherConnection dispatcherConnection, @Nullable AuthToken authTokenForDispatcher, @Nullable AuthToken authTokenForProcessor, @NotNull QueryDispatcherConnection.QueryProcessorRequestHandle handle, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, @NotNull TokenFactoryFactory tokenAuthenticationManager, String[] classpath, int maxHeapSizeMb, String queryDescription, boolean debug, boolean detailedGCLogging, boolean omitDefaultGcParameters, boolean exportQueryScope, List<RemoteQueryClassDefinition> classPushList, Map<String,String> extraEnvironmentVars, String... jvmArgs) throws RemoteQueryException
      Delegate to connectToNewProcessor(Logger, QueryDispatcherConnection, AuthToken, AuthToken, QueryDispatcherConnection.QueryProcessorRequestHandle, QueryProcessorConnection.DisconnectHandler, ConnectionMonitor.Params, TokenFactoryFactory, String[], int, int, String, boolean, boolean, boolean, boolean, List, Map, String...) with a worker priority of DEFAULT_WORKER_PRIORITY.
      Throws:
      RemoteQueryException

    • getExtraJvmArgs

      public static List<String> getExtraJvmArgs(boolean debug, int debuggerPort, String[] jvmArgs)

    • connectToExistingProcessor

      public static QueryProcessorConnection connectToExistingProcessor(@NotNull com.fishlib.io.logger.Logger log, @Nullable AuthToken authToken, int port, String hostName, String workerId, String processInfoId, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams) throws IOException
      Throws:
      IOException

    • connectToExistingProcessor

      public static QueryProcessorConnection connectToExistingProcessor(@NotNull com.fishlib.io.logger.Logger log, @Nullable AuthToken authToken, int port, String hostName, String workerId, String processInfoId, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, @NotNull TokenFactoryFactory tokenAuthenticationManager) throws IOException
      Throws:
      IOException

    • connectToExistingProcessor

      public static QueryProcessorConnection connectToExistingProcessor(@NotNull com.fishlib.io.logger.Logger log, @Nullable AuthToken authToken, int port, String hostName, int webSocketPort, String envoyPrefix, String workerId, String processInfoId, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, boolean combineWorkerConnectionStats, @NotNull TokenFactoryFactory tokenAuthenticationManager) throws IOException
      Throws:
      IOException

    • buildClassList

      public static List<RemoteQueryClassDefinition> buildClassList(Class<?>... classes) throws IOException
      Throws:
      IOException

    • buildClassList

      public static List<RemoteQueryClassDefinition> buildClassList(@Nullable List<RemoteQueryClassDefinition> classList, Class<?>... classes) throws IOException
      Throws:
      IOException

    • isConnected

      public boolean isConnected()
      Returns:
      Whether this connection is still connected to the RemoteQueryDispatcher. Note: If it's not, you'll need to create a new one and re-push any client classes needed on the server.

    • getRemoteDB

      public RemoteDatabase getRemoteDB() throws IOException
      Creates a Database object that communicates with a remote query worker. The requested worker will have the heap size specified by RemoteProcessingRequest.defaultQueryHeapMB property.
      Returns:
      a RemoteDatabase that can be used to query Deephaven
      Throws:
      IOException - if there was an error communicating with the query server

    • getRemoteDB

      public RemoteDatabase getRemoteDB(int maxHeapSizeMb) throws IOException
      Creates a Database object that communicates with a remote query worker.
      Parameters:
      maxHeapSizeMb - the worker heap size to request
      Returns:
      a RemoteDatabase that can be used to query Deephaven
      Throws:
      IOException - if there was an error communicating with the query server

    • getRemoteDB

      public RemoteDatabase getRemoteDB(int maxHeapSizeMb, @NotNull String jvmProfileName, String... extraJvmArgs) throws IOException
      Throws:
      IOException

    • getRemoteDB

      public RemoteDatabase getRemoteDB(int maxHeapSizeMb, String description, boolean debug, String... jvmArgs) throws IOException
      Throws:
      IOException

    • getRemoteDB

      public RemoteDatabase getRemoteDB(int maxHeapSizeMb, String description, boolean debug, Map<String,String> extraEnvironmentVars, String... jvmArgs) throws IOException
      Throws:
      IOException

    • getRemoteDB

      public RemoteDatabase getRemoteDB(TokenFactoryFactory tokenAuthenticationManager, int maxHeapSizeMb, String description, boolean debug, Map<String,String> extraEnvironmentVars, String... jvmArgs) throws IOException
      Throws:
      IOException

    • getRemoteDB

      public RemoteDatabase getRemoteDB(@Nullable AuthToken authTokenForDispatcher, @Nullable AuthToken authTokenForProcessor, @NotNull QueryDispatcherConnection.QueryProcessorRequestHandle handle, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, @NotNull TokenFactoryFactory tokenAuthenticationManager, String[] classpath, int maxHeapSizeMb, String queryDescription, boolean debug, boolean detailedGCLogging, List<RemoteQueryClassDefinition> classPushList, Map<String,String> extraEnvironmentVars, String... jvmArgs) throws IOException
      Throws:
      IOException

    • getRemoteDB

      public RemoteDatabase getRemoteDB(@Nullable AuthToken authTokenForDispatcher, @Nullable AuthToken authTokenForProcessor, @NotNull QueryDispatcherConnection.QueryProcessorRequestHandle handle, @Nullable QueryProcessorConnection.DisconnectHandler disconnectHandler, @Nullable io.deephaven.enterprise.comm.ConnectionMonitor.Params connectionMonitorParams, @NotNull TokenFactoryFactory tokenAuthenticationManager, String[] classpath, int maxHeapSizeMb, String queryDescription, boolean debug, boolean detailedGCLogging, boolean omitDefaultGcParameters, List<RemoteQueryClassDefinition> classPushList, Map<String,String> extraEnvironmentVars, String... jvmArgs) throws IOException
      Throws:
      IOException

    • pushClasses

      public void pushClasses(Class<?>... classes) throws IOException
      Pushes classes to the dispatcher for later use when processing queries.
      Parameters:
      classes - The classes to be pushed
      Throws:
      IOException

    • sendQuery

      public <T> T sendQuery(RemoteQuery<T> remoteQuery, int processorHeapMB, long maxDurationMS, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Does not push the query class itself, unless you add it to additionalClasses.
      Type Parameters:
      T - The return type of remoteQuery.execute()
      Parameters:
      remoteQuery - The query to execute
      processorHeapMB - The heap size (in megabytes) to request for the query processor
      maxDurationMS - Maximum duration (in millseconds) the dispatcher should expect the query to run for
      additionalClasses - For convenience, additional classes to push to the query server
      Returns:
      The result of remoteQuery.execute()
      Throws:
      IOException

    • sendQuery

      public <T> T sendQuery(RemoteQuery<T> remoteQuery, int processorHeapMB, long maxDurationMS, long workerAcquisitionTimeoutMillis, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Does not push the query class itself, unless you add it to additionalClasses.
      Parameters:
      remoteQuery - The query to execute
      processorHeapMB - The heap size (in megabytes) to request for the query processor
      maxDurationMS - Maximum duration (in millseconds) the dispatcher should expect the query to run for
      workerAcquisitionTimeoutMillis -
      additionalClasses - For convenience, additional classes to push to the query server @return The result of remoteQuery.execute()
      Throws:
      IOException

    • sendQuery

      public <T> T sendQuery(RemoteQuery<T> remoteQuery, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Does not push the query class itself, unless you add it to additionalClasses. Applies configuration defaults for maximum heap size and query duration.
      Type Parameters:
      T - The return type of remoteQuery.execute()
      Parameters:
      remoteQuery - The query to execute
      additionalClasses - For convenience, additional classes to push to the query server
      Returns:
      The result of remoteQuery.execute()
      Throws:
      IOException

    • sendQuery

      public <T> T sendQuery(RemoteQuery<T> remoteQuery, long workerAcquisitionTimeoutMillis, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Does not push the query class itself, unless you add it to additionalClasses. Applies configuration defaults for maximum heap size and query duration.
      Parameters:
      remoteQuery - The query to execute
      workerAcquisitionTimeoutMillis -
      additionalClasses - For convenience, additional classes to push to the query server @return The result of remoteQuery.execute()
      Throws:
      IOException

    • sendQuery

      public <T> T sendQuery(RemoteQuery<T> remoteQuery, RemoteProcessingRequest request, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Does not push the query class itself, unless you add it to additionalClasses.
      Type Parameters:
      T - The return type of remoteQuery.execute()
      Parameters:
      remoteQuery - The query to execute
      request - Instructions for query execution
      additionalClasses - For convenience, additional classes to push to the query server
      Returns:
      The result of remoteQuery.execute()
      Throws:
      IOException

    • sendQuery

      public <T> T sendQuery(RemoteQuery<T> remoteQuery, RemoteProcessingRequest request, long workerAcquisitionTimeoutMillis, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Does not push the query class itself, unless you add it to additionalClasses.
      Parameters:
      remoteQuery - The query to execute
      request - Instructions for query execution
      workerAcquisitionTimeoutMillis -
      additionalClasses - For convenience, additional classes to push to the query server @return The result of remoteQuery.execute()
      Throws:
      IOException

    • sendNewQuery

      public <T> T sendNewQuery(RemoteQuery<T> remoteQuery, int processorHeapMB, long maxDurationMS, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Always pushes the query class, in addition to the additionalClasses specified.
      Type Parameters:
      T - The return type of remoteQuery.execute()
      Parameters:
      remoteQuery - The query to execute
      processorHeapMB - The heap size (in megabytes) to request for the query processor
      maxDurationMS - Maximum duration (in millseconds) the dispatcher should expect the query to run for
      additionalClasses - For convenience, additional classes to push to the query server
      Returns:
      The result of remoteQuery.execute()
      Throws:
      IOException

    • sendNewQuery

      public <T> T sendNewQuery(RemoteQuery<T> remoteQuery, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Always pushes the query class, in addition to the additionalClasses specified. Applies configuration defaults for maximum heap size and query duration.
      Type Parameters:
      T - The return type of remoteQuery.execute()
      Parameters:
      remoteQuery - The query to execute
      additionalClasses - For convenience, additional classes to push to the query server
      Returns:
      The result of remoteQuery.execute()
      Throws:
      IOException

    • sendNewQuery

      public <T> T sendNewQuery(RemoteQuery<T> remoteQuery, long workerAcquisitionTimeoutMillis, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Always pushes the query class, in addition to the additionalClasses specified. Applies configuration defaults for maximum heap size and query duration.
      Parameters:
      remoteQuery - The query to execute
      workerAcquisitionTimeoutMillis -
      additionalClasses - For convenience, additional classes to push to the query server @return The result of remoteQuery.execute()
      Throws:
      IOException

    • sendNewQuery

      public <T> T sendNewQuery(RemoteQuery<T> remoteQuery, RemoteProcessingRequest request, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Always pushes the query class, in addition to the additionalClasses specified.
      Type Parameters:
      T - The return type of remoteQuery.execute()
      Parameters:
      remoteQuery - The query to execute
      request - Instructions for query execution
      additionalClasses - For convenience, additional classes to push to the query server
      Returns:
      The result of remoteQuery.execute()
      Throws:
      IOException

    • sendNewQuery

      public <T> T sendNewQuery(RemoteQuery<T> remoteQuery, RemoteProcessingRequest request, long workerAcquisitionTimeoutMillis, Class<?>... additionalClasses) throws IOException
      Requests a query processor from the dispatcher, and sends a query to the processor for execution. Always pushes the query class, in addition to the additionalClasses specified.
      Parameters:
      remoteQuery - The query to execute
      request - Instructions for query execution
      workerAcquisitionTimeoutMillis -
      additionalClasses - For convenience, additional classes to push to the query server @return The result of remoteQuery.execute()
      Throws:
      IOException

    • handleCommand

      public void handleCommand(io.deephaven.enterprise.comm.api.ClientEntry client, com.fishlib.base.Command comm)
      Specified by:
      handleCommand in interface io.deephaven.enterprise.comm.api.CommandHandler

    • handleClientDeath

      public void handleClientDeath(io.deephaven.enterprise.comm.api.ClientEntry client)
      Specified by:
      handleClientDeath in interface io.deephaven.enterprise.comm.api.CommandHandler