Package com.illumon.iris.controller

Class SimpleServerSelectionProvider

java.lang.Object

com.illumon.iris.controller.SimpleServerSelectionProvider

All Implemented Interfaces:

IServerSelectionProvider

public class SimpleServerSelectionProvider
extends Object
implements IServerSelectionProvider

Simple implementation of IServerSelectionProvider. This performs round-robin based on server availability and current usage, selecting from available server with the requested available heap, based on lowest heap utilization percentage and worker count. In its current form it is probably not suitable for production use because more intelligence is needed for small and large heap requests.

It is driven by the following properties. All properties are required for all groups, unless indicated as optional.

• PROP_ACTIVE_GROUPS specifies the names of the active groups. This drives the rest of the properties - these group names are used to derive the other property names.
• All other properties start with PROP_PREFIX followed by the group name, and the property being specified.
• PROP_SERVER_CLASS_SUFFIX specifies the server class for this group. Only servers of this class will be chosen when a server is requested. Typical values are Query and Merge. If this isn't supplied, then the value specified by the property PersistentQueryDbServerConfig.DEFAULT_SERVER_CLASS_PROP will be used.
• PROP_SERVERS_SUFFIX is optional and specifies the servers in this group. If it's not specified, all servers of the specified class will be in this group.
• PROP_MAX_HEAP_PER_WORKER_SUFFIX specifies the maximum heap in MB allowed per worker for this group. If it's not specified or is 0, then the value from PROP_DEFAULT_TOTAL_HEAP_SUFFIX will be used.
• PROP_DEFAULT_TOTAL_HEAP_SUFFIX specifies the default total heap for each server for this group. This will be updated by the controller after it gets a connection to each dispatcher.

For example:

SimpleServerSelectionProvider.ActiveGroups=AutoQuery,AutoMerge

SimpleServerSelectionProvider.Group.AutoQuery.ServerClass=Query

# No servers specified by an AutoQuery.Servers property, so all query servers will be in this group

SimpleServerSelectionProvider.Group.AutoQuery.MaxHeapMBPerWorker=32768

SimpleServerSelectionProvider.Group.AutoQuery.DefaultHeapMBPerServer=65536

SimpleServerSelectionProvider.Group.AutoMerge.ServerClass=Merge

SimpleServerSelectionProvider.Group.AutoMerge.Servers=Merge_Server_1,Merge_Server_2

SimpleServerSelectionProvider.Group.AutoMerge.MaxHeapMBPerWorker=32768

SimpleServerSelectionProvider.Group.AutoMerge.DefaultHeapMBPerServer=65536

This implementation is fully reloadable.

Nested Class Summary

Nested classes/interfaces inherited from interface com.illumon.iris.controller.IServerSelectionProvider

IServerSelectionProvider.ResourcesUnavailableException, IServerSelectionProvider.ServerGroup

Field Summary

Fields

Modifier and Type	Field	Description
static final int	NO_GROUP_PER_WORKER_HEAP_RESTRICTIONS

Constructor Summary

Constructors

Constructor	Description
SimpleServerSelectionProvider(com.fishlib.io.logger.Logger log, IrisGroupProvider irisGroupProvider)	Required constructor for the controller to create a SimpleServerSelectionProvider instance.

Method Summary

Modifier and Type	Method	Description
void	addWorkerForSerial(PersistentQueryDbServerConfig serverConfig, long serial, String processInfoId)	Add a worker for the given persistent query serial number.
String	adminCommand(UserContext userContext, String command)	Issue an administrative command to the selection provider.
PersistentQueryDbServerConfig	determineServer(String serverGroupName, UserContext userContext, int heapSizeMB, String workerKind, String description, PersistentQueryConfiguration config)	Determine the server (dispatcher) on which to request the worker.
void	generateReloadableConfigurationElement(org.jdom2.Element baseElement, com.fishlib.configuration.Configuration config)	Generate an Element with this server selection provider's configuration.
String	generateStateInformation(UserContext userContext)	Return information about the selection provider, which can be used for debugging or monitoring.
Set<IServerSelectionProvider.ServerGroup>	getServerGroups()	List the available server groups.
void	installConfiguration()	Install the verified configuration.
void	removeWorkerForSerial(PersistentQueryDbServerConfig serverConfig, long serial, String processInfoId)	Remove a worker for the given persistent query serial number.
void	setServerUsage(PersistentQueryDbServerConfig serverConfig, int usageMB, int availableMB, int workerCount)	Set the current server usage.
void	updateServerAvailability(PersistentQueryDbServerConfig serverConfig, boolean available)	Update whether a server is available or not.
void	updateServerInfo(PersistentQueryDbServerConfig serverConfig, String json)	Update the server info JSON for a server based on the dispatcher's actual configuration.
void	updateServerMaxHeapGB(PersistentQueryDbServerConfig serverConfig, double maxHeapSizeGB)	Update the max per-worker heap size in GB for a server based on the dispatcher's actual configuration.
void	updateServers()
void	updateServerTotalHeapMB(PersistentQueryDbServerConfig serverConfig, int totalHeapMB)	Update the total heap available for a server.
void	updateWorkerStatus(PersistentQueryDbServerConfig serverConfig, long serial, String processInfoId, PersistentQueryState.Status status)	Set the status of a worker for the given persistent query serial number.
void	verifyConfiguration(org.jdom2.Element element, List<PersistentQueryDbServerConfig> serverConfigs)	Given the Element generated by IServerSelectionProvider.generateReloadableConfigurationElement(org.jdom2.Element, com.fishlib.configuration.Configuration), validate and prepare the configuration for use.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

NO_GROUP_PER_WORKER_HEAP_RESTRICTIONS

public static final int NO_GROUP_PER_WORKER_HEAP_RESTRICTIONS

See Also:

• Constant Field Values

Constructor Details

SimpleServerSelectionProvider

public SimpleServerSelectionProvider(@NotNull
 com.fishlib.io.logger.Logger log,
 @Nullable
 IrisGroupProvider irisGroupProvider)

Required constructor for the controller to create a SimpleServerSelectionProvider instance.

Parameters:

log - the Logger

irisGroupProvider - the IrisGroupProvider which is used to get group membership, may be null if the provider is just being used to generate the configuration element

Method Details

generateReloadableConfigurationElement

public void generateReloadableConfigurationElement(@NotNull
 org.jdom2.Element baseElement,
 @NotNull
 com.fishlib.configuration.Configuration config)

Description copied from interface: IServerSelectionProvider

Generate an Element with this server selection provider's configuration. This element will be used to load the configuration indirectly. This element may be generated in a different process than the controller.

Specified by:

generateReloadableConfigurationElement in interface IServerSelectionProvider

Parameters:

baseElement - the Element

config - the Configuration to be used to generate the Element

verifyConfiguration

public void verifyConfiguration(@NotNull
 org.jdom2.Element element,
 List<PersistentQueryDbServerConfig> serverConfigs)
                         throws ConfigurationVerificationException

Description copied from interface: IServerSelectionProvider

Given the Element generated by IServerSelectionProvider.generateReloadableConfigurationElement(org.jdom2.Element, com.fishlib.configuration.Configuration), validate and prepare the configuration for use.

Specified by:

verifyConfiguration in interface IServerSelectionProvider

Parameters:

element - the generated Element containing the configuration

serverConfigs - the available dispatchers as determined by the controller

Throws:

ConfigurationVerificationException - if the provided element can't be used for any reason

installConfiguration

public void installConfiguration()

Description copied from interface: IServerSelectionProvider

Install the verified configuration. This will be called after IServerSelectionProvider.verifyConfiguration(org.jdom2.Element, java.util.List<com.illumon.iris.controller.PersistentQueryDbServerConfig>) is successfully called and indicates that the verified configuration should now be used.

Specified by:

installConfiguration in interface IServerSelectionProvider

updateServerTotalHeapMB

public void updateServerTotalHeapMB(@NotNull
 PersistentQueryDbServerConfig serverConfig,
 int totalHeapMB)

Description copied from interface: IServerSelectionProvider

Update the total heap available for a server. This will be determined by the controller after getting a connection to each dispatcher.

Specified by:

updateServerTotalHeapMB in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

totalHeapMB - the total available heap for the dispatcher

getServerGroups

@NotNull
public Set<IServerSelectionProvider.ServerGroup> getServerGroups()

Description copied from interface: IServerSelectionProvider

List the available server groups. Each server group will be added to ths list of servers available for query server selection.

Specified by:

getServerGroups in interface IServerSelectionProvider

Returns:

the set of available server groups

updateServers

public void updateServers()

determineServer

@NotNull
public PersistentQueryDbServerConfig determineServer(@NotNull
 String serverGroupName,
 @NotNull
 UserContext userContext,
 int heapSizeMB,
 @NotNull
 String workerKind,
 String description,
 @Nullable
 PersistentQueryConfiguration config)
                                              throws IServerSelectionProvider.ResourcesUnavailableException

Description copied from interface: IServerSelectionProvider

Determine the server (dispatcher) on which to request the worker.

Specified by:

determineServer in interface IServerSelectionProvider

Parameters:

serverGroupName - the user-chosen host or server group

userContext - the user context when available (internally-generated starts may not have a user context)

heapSizeMB - the requested heap size in MB

workerKind - the kind of worker that will be created

description - a text description of the requested worker

config - the configuration for a persistent query, or null for consoles or other non-PQ workers

Returns:

a PersistentQueryDbServerConfig that represents the server (dispatcher) on which to request the worker

Throws:

IServerSelectionProvider.ResourcesUnavailableException - if a server can't be determined

setServerUsage

public void setServerUsage(@NotNull
 PersistentQueryDbServerConfig serverConfig,
 int usageMB,
 int availableMB,
 int workerCount)

Description copied from interface: IServerSelectionProvider

Set the current server usage. This may be on an initial dispatcher connection or after a reconnection, or whenever the utilization changes.

Specified by:

setServerUsage in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

usageMB - the usage in MB

availableMB - the server's availability as reported by the dispatcher

workerCount - the current number of workers running on that server

addWorkerForSerial

public void addWorkerForSerial(@NotNull
 PersistentQueryDbServerConfig serverConfig,
 long serial,
 String processInfoId)

Description copied from interface: IServerSelectionProvider

Add a worker for the given persistent query serial number.

The server selection provider may use the worker count per PQ to more evenly distribute replicas across the cluster so that the failure of one query node is less likely to make all replicas of a query unavailable.

Specified by:

addWorkerForSerial in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

serial - the serial number of the persistent query that is starting a new worker

processInfoId - the process info ID of the worker

removeWorkerForSerial

public void removeWorkerForSerial(@NotNull
 PersistentQueryDbServerConfig serverConfig,
 long serial,
 String processInfoId)

Description copied from interface: IServerSelectionProvider

Remove a worker for the given persistent query serial number.

The server selection provider may use the worker count per PQ to more evenly distribute replicas across the cluster so that the failure of one query node is less likely to make all replicas of a query unavailable.

Specified by:

removeWorkerForSerial in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

serial - the serial number of the persistent query that has removed a worker

processInfoId - the process info ID of the worker

updateWorkerStatus

public void updateWorkerStatus(PersistentQueryDbServerConfig serverConfig,
 long serial,
 @NotNull
 String processInfoId,
 @NotNull
 PersistentQueryState.Status status)

Description copied from interface: IServerSelectionProvider

Set the status of a worker for the given persistent query serial number.

The server selection provider may use this information to `guess` about server health.

Specified by:

updateWorkerStatus in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

serial - the serial number of the persistent query that has removed a worker

processInfoId - the process info ID of the worker

status - the most recent status transition for the worker

updateServerAvailability

public void updateServerAvailability(@NotNull
 PersistentQueryDbServerConfig serverConfig,
 boolean available)

Description copied from interface: IServerSelectionProvider

Update whether a server is available or not.

Specified by:

updateServerAvailability in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

available - if true, the server is available; if false, it is not

updateServerMaxHeapGB

public void updateServerMaxHeapGB(PersistentQueryDbServerConfig serverConfig,
 double maxHeapSizeGB)

Description copied from interface: IServerSelectionProvider

Update the max per-worker heap size in GB for a server based on the dispatcher's actual configuration.

Specified by:

updateServerMaxHeapGB in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

maxHeapSizeGB - the new maximum size

updateServerInfo

public void updateServerInfo(PersistentQueryDbServerConfig serverConfig,
 String json)

Description copied from interface: IServerSelectionProvider

Update the server info JSON for a server based on the dispatcher's actual configuration.

Specified by:

updateServerInfo in interface IServerSelectionProvider

Parameters:

serverConfig - the server configuration

json - the server's JSON server info configuration

generateStateInformation

public String generateStateInformation(@NotNull
 UserContext userContext)

Description copied from interface: IServerSelectionProvider

Return information about the selection provider, which can be used for debugging or monitoring.

The state format is defined by the selection provider, but the suggested format is JSON for simple parsing with jq or other tools.

Specified by:

generateStateInformation in interface IServerSelectionProvider

Parameters:

userContext - the user requesting the state

Returns:

a string representation of the internal state of this selection provider.

adminCommand

public String adminCommand(@NotNull
 UserContext userContext,
 @NotNull
 String command)

Description copied from interface: IServerSelectionProvider

Issue an administrative command to the selection provider.

Specified by:

adminCommand in interface IServerSelectionProvider

Parameters:

userContext - the user requesting the command

command - the administrative command

Returns:

a string representation of the result