Package io.deephaven.engine.table.impl.util

Interface JobScheduler

All Known Implementing Classes:
ImmediateJobScheduler, OperationInitializerJobScheduler, UpdateGraphJobScheduler
public interface JobScheduler
An interface for submitting jobs to be executed. Submitted jobs may be executed on the current thread, or in separate threads (thus allowing true parallelism). Performance metrics are accumulated for all executions off the current thread for inclusion in overall task metrics.

  • Field Details

    • DEFAULT_CONTEXT

      static final JobScheduler.JobThreadContext DEFAULT_CONTEXT

    • DEFAULT_CONTEXT_FACTORY

      static final Supplier<JobScheduler.JobThreadContext> DEFAULT_CONTEXT_FACTORY

    • UNREPORTABLE_JOB_ERROR

      static final Exception UNREPORTABLE_JOB_ERROR
      Delivered when a job fails with an Error and even the wrapper for it cannot be allocated, which is to say when the heap is exhausted — the very failure this path exists for. Allocated once, when this interface is initialized, so that delivering a failure never depends on being able to allocate.

  • Method Details

    • asDeliverableException

      static Exception asDeliverableException(@NotNull @NotNull Throwable throwable)
      Convert a Throwable that escaped a scheduled job into something the Consumer<Exception> error handlers used throughout the scheduler can accept. Exceptions pass through unchanged; an Error — an OutOfMemoryError, in practice — is wrapped, so that the thread waiting on the job's completion fails with a diagnostic instead of waiting forever for a completion that cannot happen.

      This never throws. The wrapper carries no stack trace of its own: filling one in is the largest allocation here, and the stack that matters belongs to the Error, which is kept as the cause. Should even that allocation fail, UNREPORTABLE_JOB_ERROR is delivered instead — a caller that fails without a diagnostic is still far better than one that waits forever.

      Parameters:
      throwable - the Throwable that escaped the job
      Returns:
      throwable itself if it is an Exception, otherwise a wrapper holding it as its cause

    • submit

      void submit(ExecutionContext executionContext, Runnable runnable, LogOutputAppendable description, Consumer<Exception> onError)
      Cause runnable to be executed.
      Parameters:
      executionContext - the execution context to run it under
      runnable - the runnable to execute
      description - a description for logging
      onError - a routine to call if an exception occurs while running runnable

    • getAccumulatedPerformance

      BasePerformanceEntry getAccumulatedPerformance()
      The performance statistics of all runnables that have been completed off-thread, or null if all were executed in the current thread.

      When initializing an operation, the OperationInitializerJobScheduler executes the completion callback as part of another task. Therefore, you must not read the accumulated performance from the completion callback when initializing an operation; as it could miss data from some of the tasks. Furthermore, even though the completion callback identifies the result as ready does not mean that the completion callback has actually completed. To guard against this the OperationInitializerJobScheduler waits for all jobs to be complete before returning the BasePerformanceEntry. Therefore, if you call this from a completion callback, then the operation will hang.

    • threadCount

      int threadCount()
      How many threads exist in the job scheduler? The job submitters can use this value to determine how many sub-jobs to split work into.

    • iterateParallel

      @FinalDefault default <CONTEXT_TYPE extends JobScheduler.JobThreadContext> void iterateParallel(@Nullable @Nullable ExecutionContext executionContext, @Nullable @Nullable LogOutputAppendable description, @NotNull @NotNull Supplier<CONTEXT_TYPE> taskThreadContextFactory, int start, int count, @NotNull @NotNull JobScheduler.IterateAction<CONTEXT_TYPE> action, @NotNull @NotNull Runnable onComplete, @NotNull @NotNull Runnable cleanup, @NotNull @NotNull Consumer<Exception> onError)
      Provides a mechanism to iterate over a range of values in parallel using the JobScheduler
      Parameters:
      executionContext - the execution context for this task
      description - the description to use for logging
      taskThreadContextFactory - the factory that supplies contexts for the threads handling the sub-tasks
      start - the integer value from which to start iterating
      count - the number of times this task should be called
      action - the task to perform, the current iteration index is provided as a parameter
      onComplete - this will be called when all iterations are complete
      cleanup - called after onComplete successfully returns. If the invocation of the cleanup throws an exception, onError will not be called.
      onError - error handler for the scheduler to use while iterating, or if onComplete throws an exception.

    • iterateParallel

      @FinalDefault default <CONTEXT_TYPE extends JobScheduler.JobThreadContext> void iterateParallel(@Nullable @Nullable ExecutionContext executionContext, @Nullable @Nullable LogOutputAppendable description, @NotNull @NotNull Supplier<CONTEXT_TYPE> taskThreadContextFactory, int start, int count, @NotNull @NotNull JobScheduler.IterateResumeAction<CONTEXT_TYPE> action, @NotNull @NotNull Runnable onComplete, @NotNull @NotNull Runnable cleanup, @NotNull @NotNull Consumer<Exception> onError)
      Provides a mechanism to iterate over a range of values in parallel using the JobScheduler. The advantage to using this over the other method is the resumption callable on action that will trigger the next execution. This allows the next iteration and the completion runnable to be delayed until dependent asynchronous serial or parallel scheduler jobs have completed.
      Parameters:
      executionContext - the execution context for this task
      description - the description to use for logging
      taskThreadContextFactory - the factory that supplies contexts for the tasks
      start - the integer value from which to start iterating
      count - the number of times this task should be called
      action - the task to perform, the current iteration index and a resume Runnable are parameters
      onComplete - this will be called when all iterations are complete
      cleanup - called after onComplete successfully returns. If the invocation of the cleanup throws an exception, onError will not be called.
      onError - error handler for the scheduler to use while iterating, or if onComplete throws an exception.

    • iterateSerial

      @FinalDefault default <CONTEXT_TYPE extends JobScheduler.JobThreadContext> void iterateSerial(@Nullable @Nullable ExecutionContext executionContext, @Nullable @Nullable LogOutputAppendable description, @NotNull @NotNull Supplier<CONTEXT_TYPE> taskThreadContextFactory, int start, int count, @NotNull @NotNull JobScheduler.IterateResumeAction<CONTEXT_TYPE> action, @NotNull @NotNull Runnable onComplete, @NotNull @NotNull Runnable cleanup, @NotNull @NotNull Consumer<Exception> onError)
      Provides a mechanism to iterate over a range of values serially using the JobScheduler. The advantage to using this over a simple iteration is the resumption callable on action that will trigger the next execution. This allows the next iteration and the completion runnable to be delayed until dependent asynchronous serial or parallel scheduler jobs have completed.
      Parameters:
      executionContext - the execution context for this task
      description - the description to use for logging
      taskThreadContextFactory - the factory that supplies contexts for the tasks
      start - the integer value from which to start iterating
      count - the number of times this task should be called
      action - the task to perform, the current iteration index and a resume Runnable are parameters
      onComplete - this will be called when all iterations are complete
      cleanup - called after onComplete successfully returns. If the invocation of the cleanup throws an exception, onError will not be called.
      onError - error handler for the scheduler to use while iterating, or if onComplete throws an exception.