Package io.deephaven.engine.table.impl.dataindex

Class DataIndexPushdownManager

java.lang.Object

io.deephaven.engine.table.impl.dataindex.DataIndexPushdownManager

All Implemented Interfaces:

PushdownFilterMatcher, PushdownPredicateManager

public class DataIndexPushdownManager
extends Object
implements PushdownPredicateManager

A PushdownPredicateManager that supports pushdown against a DataIndex. This manager can wrap an existing PushdownFilterMatcher to allow efficient fallback if the DataIndex pushdown is not effective or when the cost ceiling is too low to use the DataIndex.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DataIndexPushdownManager.DataIndexPushdownContext

Method Summary

Modifier and Type	Method	Description
void	estimatePushdownFilterCost(@NotNull WhereFilter filter, @NotNull RowSet selection, boolean usePrev, @NotNull PushdownFilterContext context, @NotNull JobScheduler jobScheduler, @NotNull LongConsumer onComplete, @NotNull Consumer<Exception> onError)	Estimate the cost of pushing down the next pushdown filter.
PushdownFilterContext	makePushdownFilterContext(@NotNull WhereFilter filter, @NotNull List<ColumnSource<?>> filterSources)	Create a pushdown filter context for this entity.
void	pushdownFilter(@NotNull WhereFilter filter, @NotNull RowSet selection, boolean usePrev, @NotNull PushdownFilterContext context, long costCeiling, @NotNull JobScheduler jobScheduler, @NotNull Consumer<PushdownResult> onComplete, @NotNull Consumer<Exception> onError)	Push down the given filter to the underlying table and pass the result to the consumer.
static PushdownFilterMatcher	wrap(@NotNull DataIndex dataIndex, PushdownFilterMatcher wrapped)	Create new DataIndexPushdownManager for a DataIndex, wrapping a PushdownFilterMatcher to be used when the index would not be efficient.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

estimatePushdownFilterCost

public void estimatePushdownFilterCost(@NotNull
 @NotNull WhereFilter filter,
 @NotNull
 @NotNull RowSet selection,
 boolean usePrev,
 @NotNull
 @NotNull PushdownFilterContext context,
 @NotNull
 @NotNull JobScheduler jobScheduler,
 @NotNull
 @NotNull LongConsumer onComplete,
 @NotNull
 @NotNull Consumer<Exception> onError)

Description copied from interface: PushdownFilterMatcher

Estimate the cost of pushing down the next pushdown filter. This returns a unitless value to compare the cost of executing different filters. Common costs are listed in PushdownResult (such as PushdownResult.METADATA_STATS_COST) and should be used as a baseline for estimating the cost of newly implemented pushdown operations.

A no-op implementation should simply complete with Long.MAX_VALUE.

Specified by:

estimatePushdownFilterCost in interface PushdownFilterMatcher

Parameters:

filter - The filter to test.

selection - The set of rows to tests.

usePrev - Whether to use the previous result

context - The PushdownFilterContext to use for the pushdown operation.

jobScheduler - The job scheduler to use for scheduling child jobs

onComplete - Consumer of the estimated cost of the pushdown operation. 9223372036854775807L indicates that the filter cannot be pushed down.

onError - Consumer of any exceptions that occur during the estimate operation

pushdownFilter

public void pushdownFilter(@NotNull
 @NotNull WhereFilter filter,
 @NotNull
 @NotNull RowSet selection,
 boolean usePrev,
 @NotNull
 @NotNull PushdownFilterContext context,
 long costCeiling,
 @NotNull
 @NotNull JobScheduler jobScheduler,
 @NotNull
 @NotNull Consumer<PushdownResult> onComplete,
 @NotNull
 @NotNull Consumer<Exception> onError)

Description copied from interface: PushdownFilterMatcher

Push down the given filter to the underlying table and pass the result to the consumer. This method is expected to execute all pushdown filter steps that are greater than PushdownFilterContext.executedFilterCost() and less than or equal to costCeiling.

The resulting PushdownResult (to onComplete) must only contain rows from selection. The match row set are rows that are guaranteed to match. The implicitly "missing" rows selection - match - maybeMatch are rows that are guaranteed to not match. The remaining maybe match row set are rows that may, or may not, match. The pushdown result ownership passes to onComplete.

A no-op implementation should simply complete with PushdownResult.maybeMatch(selection).

Specified by:

pushdownFilter in interface PushdownFilterMatcher

Parameters:

filter - The filter to apply.

selection - The set of rows to test.

usePrev - Whether to use the previous result

context - The PushdownFilterContext to use for the pushdown operation.

costCeiling - Execute all possible filters with a cost less than or equal this value.

jobScheduler - The job scheduler to use for scheduling child jobs

onComplete - Consumer of the output rowsets for added and modified rows that pass the filter

onError - Consumer of any exceptions that occur during the pushdown operation

makePushdownFilterContext

public PushdownFilterContext makePushdownFilterContext(@NotNull
 @NotNull WhereFilter filter,
 @NotNull
 @NotNull List<ColumnSource<?>> filterSources)

Description copied from interface: PushdownFilterMatcher

Create a pushdown filter context for this entity.

Specified by:

makePushdownFilterContext in interface PushdownFilterMatcher

Parameters:

filter - the filter to use while making the context

filterSources - the column sources that match the filter column names

Returns:

the created filter context

wrap

public static PushdownFilterMatcher wrap(@NotNull
 @NotNull DataIndex dataIndex,
 PushdownFilterMatcher wrapped)

Create new DataIndexPushdownManager for a DataIndex, wrapping a PushdownFilterMatcher to be used when the index would not be efficient.

Parameters:

dataIndex - the DataIndex to use for pushdown

wrapped - the wrapped PushdownFilterMatcher

Returns:

a new PushdownFilterMatcher supporting DataIndex pushdown, or null if pushdown is not supported