Package com.illumon.iris.db.v2.by

Interface IterativeChunkedAggregationOperator

All Known Implementing Classes:

AddOnlyFirstOrLastChunkedOperator, BigDecimalChunkedSumOperator, BigIntegerChunkedSumOperator, BooleanChunkedSumOperator, ByChunkedOperator, ByExternalChunkedOperator, ByteAddOnlySortedFirstOrLastChunkedOperator, ByteChunkedCountDistinctOperator, ByteChunkedDistinctOperator, ByteChunkedSumOperator, ByteChunkedUniqueOperator, ByteRollupCountDistinctOperator, ByteRollupDistinctOperator, ByteRollupUniqueOperator, CharAddOnlySortedFirstOrLastChunkedOperator, CharChunkedCountDistinctOperator, CharChunkedDistinctOperator, CharChunkedSumOperator, CharChunkedUniqueOperator, CharRollupCountDistinctOperator, CharRollupDistinctOperator, CharRollupUniqueOperator, DoubleAddOnlySortedFirstOrLastChunkedOperator, DoubleChunkedCountDistinctOperator, DoubleChunkedDistinctOperator, DoubleChunkedUniqueOperator, DoubleRollupCountDistinctOperator, DoubleRollupDistinctOperator, DoubleRollupUniqueOperator, FirstOrLastChunkedOperator, FloatAddOnlySortedFirstOrLastChunkedOperator, FloatChunkedCountDistinctOperator, FloatChunkedDistinctOperator, FloatChunkedUniqueOperator, FloatRollupCountDistinctOperator, FloatRollupDistinctOperator, FloatRollupUniqueOperator, FreezeByCountOperator, FreezeByOperator, IntAddOnlySortedFirstOrLastChunkedOperator, IntChunkedCountDistinctOperator, IntChunkedDistinctOperator, IntChunkedSumOperator, IntChunkedUniqueOperator, IntRollupCountDistinctOperator, IntRollupDistinctOperator, IntRollupUniqueOperator, LongAddOnlySortedFirstOrLastChunkedOperator, LongChunkedCountDistinctOperator, LongChunkedDistinctOperator, LongChunkedSumOperator, LongChunkedUniqueOperator, LongRollupCountDistinctOperator, LongRollupDistinctOperator, LongRollupUniqueOperator, ObjectAddOnlySortedFirstOrLastChunkedOperator, ObjectChunkedCountDistinctOperator, ObjectChunkedDistinctOperator, ObjectChunkedUniqueOperator, ObjectRollupCountDistinctOperator, ObjectRollupDistinctOperator, ObjectRollupUniqueOperator, ShortAddOnlySortedFirstOrLastChunkedOperator, ShortChunkedCountDistinctOperator, ShortChunkedDistinctOperator, ShortChunkedSumOperator, ShortChunkedUniqueOperator, ShortRollupCountDistinctOperator, ShortRollupDistinctOperator, ShortRollupUniqueOperator, SortedFirstOrLastChunkedOperator, SsmChunkedMinMaxOperator, SsmChunkedPercentileOperator, StaticFirstOrLastChunkedOperator, TDigestPercentileOperator, UntrackedLastChunkedOperator

public interface IterativeChunkedAggregationOperator

A chunked, iterative operator that processes indices and/or data from one input column to produce one or more output columns.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static interface	IterativeChunkedAggregationOperator.BucketedContext	Context interface for bucketed operator updates.
static interface	IterativeChunkedAggregationOperator.SingletonContext	Context interface for singleton (that is, one aggregation state) operator updates.

Field Summary

Fields

Modifier and Type	Field	Description
static final IterativeChunkedAggregationOperator[]	ZERO_LENGTH_ITERATIVE_CHUNKED_AGGREGATION_OPERATOR_ARRAY

Method Summary

Modifier and Type	Method	Description
void	addChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Aggregate a chunk of data into the result columns.
boolean	addChunk(IterativeChunkedAggregationOperator.SingletonContext context, int chunkSize, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)	Aggregate a chunk of data into the result columns.
default boolean	addIndex(IterativeChunkedAggregationOperator.SingletonContext context, ReadOnlyIndex index, long destination)
void	ensureCapacity(long tableSize)	Ensure that this operator can handle destinations up to tableSize - 1.
Map<String,? extends ColumnSource<?>>	getResultColumns()	Return a map of result columns produced by this operator.
default UnaryOperator<ModifiedColumnSet>	initializeRefreshing(@NotNull QueryTable resultTable, @NotNull LivenessReferent aggregationUpdateListener)	Initialize refreshing result support for this operator.
default IterativeChunkedAggregationOperator.BucketedContext	makeBucketedContext(int size)	Make a IterativeChunkedAggregationOperator.BucketedContext suitable for this operator if necessary.
default IterativeChunkedAggregationOperator.SingletonContext	makeSingletonContext(int size)	Make a IterativeChunkedAggregationOperator.SingletonContext suitable for this operator if necessary.
default void	modifyChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values.
default boolean	modifyChunk(IterativeChunkedAggregationOperator.SingletonContext context, int chunkSize, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, long destination)	Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values.
default void	modifyIndices(IterativeChunkedAggregationOperator.BucketedContext context, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).
default boolean	modifyIndices(IterativeChunkedAggregationOperator.SingletonContext context, LongChunk<? extends Attributes.KeyIndices> indices, long destination)	Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).
default void	propagateFailure(@NotNull Throwable originalException, UpdatePerformanceTracker.Entry sourceEntry)	Called on error to propagate listener failure to this operator.
default void	propagateInitialState(@NotNull QueryTable resultTable)	Perform any internal state keeping needed for destinations that were added during initialization.
default void	propagateUpdates(ShiftAwareListener.Update downstream, @NotNull ReadOnlyIndex newDestinations)	Perform any internal state keeping needed for destinations that were added (went from 0 keys to > 0), removed (went from > 0 keys to 0), or modified (keys added or removed, or keys modified) by this iteration.
void	removeChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Remove a chunk of data previously aggregated into the result columns.
boolean	removeChunk(IterativeChunkedAggregationOperator.SingletonContext context, int chunkSize, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)	Remove a chunk of data previously aggregated into the result columns.
default boolean	requiresIndices()	Whether the operator requires indices.
default void	resetForStep(ShiftAwareListener.Update upstream)	Reset any per-step internal state.
default void	shiftChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> preShiftIndices, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Called with shifted indices when requiresIndices() returns true, including shifted same-slot modifies.
default boolean	shiftChunk(IterativeChunkedAggregationOperator.SingletonContext context, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> preShiftIndices, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, long destination)	Shift a chunk of data previously aggregated into the result columns, including shifted same-slot modifies..
void	startTrackingPrevValues()	Called after initialization; when the operator's result columns must have previous tracking enabled.
default boolean	unchunkedIndex()	Whether the operator can deal with an unchunked Index more efficiently than a chunked index.

Field Details

ZERO_LENGTH_ITERATIVE_CHUNKED_AGGREGATION_OPERATOR_ARRAY

static final IterativeChunkedAggregationOperator[] ZERO_LENGTH_ITERATIVE_CHUNKED_AGGREGATION_OPERATOR_ARRAY

Method Details

addChunk

void addChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Aggregate a chunk of data into the result columns.

Parameters:

context - the operator-specific context

values - a chunk of values to aggregate

inputIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

removeChunk

void removeChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Remove a chunk of data previously aggregated into the result columns.

Parameters:

context - the operator-specific context

values - a chunk of values that have been previously aggregated.

inputIndices - the input indices, in pre-shift space

destinations - the destinations in resultColumn to remove the values from, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

modifyChunk

default void modifyChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values. Never includes modifies that have been shifted if requiresIndices() returns true - those are handled in shiftChunk(BucketedContext, Chunk, Chunk, LongChunk, LongChunk, IntChunk, IntChunk, IntChunk, WritableBooleanChunk).

Parameters:

context - the operator-specific context

previousValues - a chunk of values that have been previously aggregated

newValues - a chunk of values to aggregate

postShiftIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to remove the values from, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

shiftChunk

default void shiftChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> preShiftIndices,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Called with shifted indices when requiresIndices() returns true, including shifted same-slot modifies.

Parameters:

context - the operator-specific context

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

preShiftIndices - the input indices, in pre-shift space

postShiftIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

modifyIndices

default void modifyIndices(IterativeChunkedAggregationOperator.BucketedContext context,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).

Parameters:

context - the operator-specific context

inputIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

addChunk

boolean addChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 int chunkSize,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 long destination)

Aggregate a chunk of data into the result columns.

Parameters:

context - the operator-specific context

chunkSize - the size of the addition

values - the values to aggregate

inputIndices - the input indices, in post-shift space

destination - the destination in the result columns

Returns:

true if the state was modified, false otherwise

removeChunk

boolean removeChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 int chunkSize,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 long destination)

Remove a chunk of data previously aggregated into the result columns.

Parameters:

context - the operator-specific context

chunkSize - the size of the removal

values - the values to remove from the aggregation

inputIndices - the input indices, in pre-shift space

destination - the destination in the result columns

Returns:

true if the state was modified, false otherwise

modifyChunk

default boolean modifyChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 int chunkSize,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 long destination)

Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values. Never includes modifies that have been shifted if requiresIndices() returns true - those are handled in shiftChunk(SingletonContext, Chunk, Chunk, LongChunk, LongChunk, long).

Parameters:

context - the operator-specific context

chunkSize - the size of the modification

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

postShiftIndices - the input indices, in post-shift space

Returns:

true if the state was modified, false otherwise

shiftChunk

default boolean shiftChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> preShiftIndices,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 long destination)

Shift a chunk of data previously aggregated into the result columns, including shifted same-slot modifies..

Parameters:

context - the operator-specific context

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

preShiftIndices - the input indices, in pre-shift space

postShiftIndices - the input indices, in post-shift space

destination - the destination in the result columns

Returns:

true if the result should be considered modified

modifyIndices

default boolean modifyIndices(IterativeChunkedAggregationOperator.SingletonContext context,
 LongChunk<? extends Attributes.KeyIndices> indices,
 long destination)

Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).

Parameters:

context - the operator-specific context

indices - the modified indices for a given destination, in post-shift space

destination - the destination that was modified

Returns:

true if the result should be considered modified

requiresIndices

default boolean requiresIndices()

Whether the operator requires indices. This implies that the operator must process shifts (i.e. shiftChunk(com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator.BucketedContext, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkPositions>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkLengths>, com.illumon.iris.db.v2.sources.chunk.WritableBooleanChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.Values>)), and must observe modifications even when its input columns (if any) are not modified (i.e. modifyIndices(com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator.BucketedContext, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkPositions>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkLengths>, com.illumon.iris.db.v2.sources.chunk.WritableBooleanChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.Values>)).

Returns:

true if the operator requires indices, false otherwise

unchunkedIndex

default boolean unchunkedIndex()

Whether the operator can deal with an unchunked Index more efficiently than a chunked index.

Returns:

true if the operator can deal with unchunked indices, false otherwise

addIndex

default boolean addIndex(IterativeChunkedAggregationOperator.SingletonContext context,
 ReadOnlyIndex index,
 long destination)

ensureCapacity

void ensureCapacity(long tableSize)

Ensure that this operator can handle destinations up to tableSize - 1.

Parameters:

tableSize - the new size of the table

getResultColumns

Map<String,? extends ColumnSource<?>> getResultColumns()

Return a map of result columns produced by this operator.

Returns:

a map of name to columns for the result table

propagateInitialState

default void propagateInitialState(@NotNull
 @NotNull QueryTable resultTable)

Perform any internal state keeping needed for destinations that were added during initialization.

Parameters:

resultTable - The result QueryTable after initialization

startTrackingPrevValues

void startTrackingPrevValues()

Called after initialization; when the operator's result columns must have previous tracking enabled.

initializeRefreshing

default UnaryOperator<ModifiedColumnSet> initializeRefreshing(@NotNull
 @NotNull QueryTable resultTable,
 @NotNull
 @NotNull LivenessReferent aggregationUpdateListener)

Initialize refreshing result support for this operator. As a side effect, make a factory method for converting upstream modified column sets to result modified column sets, to be invoked whenever this operator reports a modification in order to determine the operator's contribution to the final result modified column set.

Parameters:

resultTable - The result QueryTable after initialization

aggregationUpdateListener - The aggregation update listener, which may be needed for referential integrity

Returns:

A factory that produces a result modified column set from the upstream modified column set

resetForStep

default void resetForStep(@NotNull
 ShiftAwareListener.Update upstream)

Reset any per-step internal state. Note that the arguments to this method should not be mutated in any way.

Parameters:

upstream - The upstream ShiftAwareListener.Update

propagateUpdates

default void propagateUpdates(@NotNull
 ShiftAwareListener.Update downstream,
 @NotNull
 @NotNull ReadOnlyIndex newDestinations)

Perform any internal state keeping needed for destinations that were added (went from 0 keys to > 0), removed (went from > 0 keys to 0), or modified (keys added or removed, or keys modified) by this iteration. Note that the arguments to this method should not be mutated in any way.

Parameters:

downstream - The downstream ShiftAwareListener.Update (which does not have its ModifiedColumnSet finalized yet)

newDestinations - New destinations added on this update

propagateFailure

default void propagateFailure(@NotNull
 @NotNull Throwable originalException,
 @NotNull
 UpdatePerformanceTracker.Entry sourceEntry)

Called on error to propagate listener failure to this operator.

Parameters:

originalException - The error Throwable

sourceEntry - The UpdatePerformanceTracker.Entry for the failed listener

makeBucketedContext

default IterativeChunkedAggregationOperator.BucketedContext makeBucketedContext(int size)

Make a IterativeChunkedAggregationOperator.BucketedContext suitable for this operator if necessary.

Parameters:

size - The maximum size of input chunks that will be used with the result context

Returns:

A new IterativeChunkedAggregationOperator.BucketedContext, or null if none is necessary

makeSingletonContext

default IterativeChunkedAggregationOperator.SingletonContext makeSingletonContext(int size)

Make a IterativeChunkedAggregationOperator.SingletonContext suitable for this operator if necessary.

Parameters:

size - The maximum size of input chunks that will be used with the result context

Returns:

A new IterativeChunkedAggregationOperator.SingletonContext, or null if none is necessary