Package com.illumon.iris.db.v2.by

Interface IterativeChunkedAggregationOperator

All Known Implementing Classes:

AddOnlyFirstOrLastChunkedOperator, BigDecimalChunkedSumOperator, BigIntegerChunkedSumOperator, BooleanChunkedSumOperator, ByChunkedOperator, ByExternalChunkedOperator, ByteAddOnlySortedFirstOrLastChunkedOperator, ByteChunkedCountDistinctOperator, ByteChunkedDistinctOperator, ByteChunkedSumOperator, ByteChunkedUniqueOperator, ByteRollupCountDistinctOperator, ByteRollupDistinctOperator, ByteRollupUniqueOperator, CharAddOnlySortedFirstOrLastChunkedOperator, CharChunkedCountDistinctOperator, CharChunkedDistinctOperator, CharChunkedSumOperator, CharChunkedUniqueOperator, CharRollupCountDistinctOperator, CharRollupDistinctOperator, CharRollupUniqueOperator, DoubleAddOnlySortedFirstOrLastChunkedOperator, DoubleChunkedCountDistinctOperator, DoubleChunkedDistinctOperator, DoubleChunkedUniqueOperator, DoubleRollupCountDistinctOperator, DoubleRollupDistinctOperator, DoubleRollupUniqueOperator, FirstOrLastChunkedOperator, FloatAddOnlySortedFirstOrLastChunkedOperator, FloatChunkedCountDistinctOperator, FloatChunkedDistinctOperator, FloatChunkedUniqueOperator, FloatRollupCountDistinctOperator, FloatRollupDistinctOperator, FloatRollupUniqueOperator, FreezeByCountOperator, FreezeByOperator, IntAddOnlySortedFirstOrLastChunkedOperator, IntChunkedCountDistinctOperator, IntChunkedDistinctOperator, IntChunkedSumOperator, IntChunkedUniqueOperator, IntRollupCountDistinctOperator, IntRollupDistinctOperator, IntRollupUniqueOperator, LongAddOnlySortedFirstOrLastChunkedOperator, LongChunkedCountDistinctOperator, LongChunkedDistinctOperator, LongChunkedSumOperator, LongChunkedUniqueOperator, LongRollupCountDistinctOperator, LongRollupDistinctOperator, LongRollupUniqueOperator, ObjectAddOnlySortedFirstOrLastChunkedOperator, ObjectChunkedCountDistinctOperator, ObjectChunkedDistinctOperator, ObjectChunkedUniqueOperator, ObjectRollupCountDistinctOperator, ObjectRollupDistinctOperator, ObjectRollupUniqueOperator, ShortAddOnlySortedFirstOrLastChunkedOperator, ShortChunkedCountDistinctOperator, ShortChunkedDistinctOperator, ShortChunkedSumOperator, ShortChunkedUniqueOperator, ShortRollupCountDistinctOperator, ShortRollupDistinctOperator, ShortRollupUniqueOperator, SortedFirstOrLastChunkedOperator, SsmChunkedMinMaxOperator, SsmChunkedPercentileOperator, StaticFirstOrLastChunkedOperator, TDigestPercentileOperator, UntrackedLastChunkedOperator

public interface IterativeChunkedAggregationOperator

A chunked, iterative operator that processes indices and/or data from one input column to produce one or more output columns.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static interface	IterativeChunkedAggregationOperator.BucketedContext	Context interface for bucketed operator updates.
static interface	IterativeChunkedAggregationOperator.SingletonContext	Context interface for singleton (that is, one aggregation state) operator updates.

Field Summary

Fields

Modifier and Type	Field	Description
static final IterativeChunkedAggregationOperator[]	ZERO_LENGTH_ITERATIVE_CHUNKED_AGGREGATION_OPERATOR_ARRAY

Method Summary

Modifier and Type	Method	Description
void	addChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Aggregate a chunk of data into the result columns.
boolean	addChunk(IterativeChunkedAggregationOperator.SingletonContext context, int chunkSize, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)	Aggregate a chunk of data into the result columns.
default boolean	addIndex(IterativeChunkedAggregationOperator.SingletonContext context, ReadOnlyIndex index, long destination)
void	ensureCapacity(long tableSize)	Ensure that this operator can handle destinations up to tableSize - 1.
Map<String,? extends ColumnSource<?>>	getResultColumns()	Return a map of result columns produced by this operator.
default UnaryOperator<ModifiedColumnSet>	initializeRefreshing(QueryTable resultTable, LivenessReferent aggregationUpdateListener)	Initialize refreshing result support for this operator.
default IterativeChunkedAggregationOperator.BucketedContext	makeBucketedContext(int size)	Make a IterativeChunkedAggregationOperator.BucketedContext suitable for this operator if necessary.
default IterativeChunkedAggregationOperator.SingletonContext	makeSingletonContext(int size)	Make a IterativeChunkedAggregationOperator.SingletonContext suitable for this operator if necessary.
default void	modifyChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values.
default boolean	modifyChunk(IterativeChunkedAggregationOperator.SingletonContext context, int chunkSize, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, long destination)	Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values.
default void	modifyIndices(IterativeChunkedAggregationOperator.BucketedContext context, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).
default boolean	modifyIndices(IterativeChunkedAggregationOperator.SingletonContext context, LongChunk<? extends Attributes.KeyIndices> indices, long destination)	Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).
default void	propagateFailure(Throwable originalException, UpdatePerformanceTracker.Entry sourceEntry)	Called on error to propagate listener failure to this operator.
default void	propagateInitialState(QueryTable resultTable)	Perform any internal state keeping needed for destinations that were added during initialization.
default void	propagateUpdates(ShiftAwareListener.Update downstream, ReadOnlyIndex newDestinations)	Perform any internal state keeping needed for destinations that were added (went from 0 keys to > 0), removed (went from > 0 keys to 0), or modified (keys added or removed, or keys modified) by this iteration.
void	removeChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Remove a chunk of data previously aggregated into the result columns.
boolean	removeChunk(IterativeChunkedAggregationOperator.SingletonContext context, int chunkSize, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)	Remove a chunk of data previously aggregated into the result columns.
default boolean	requiresIndices()	Whether the operator requires indices.
default void	resetForStep(ShiftAwareListener.Update upstream)	Reset any per-step internal state.
default void	shiftChunk(IterativeChunkedAggregationOperator.BucketedContext context, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> preShiftIndices, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Called with shifted indices when requiresIndices() returns true, including shifted same-slot modifies.
default boolean	shiftChunk(IterativeChunkedAggregationOperator.SingletonContext context, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> preShiftIndices, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, long destination)	Shift a chunk of data previously aggregated into the result columns, including shifted same-slot modifies..
void	startTrackingPrevValues()	Called after initialization; when the operator's result columns must have previous tracking enabled.
default boolean	unchunkedIndex()	Whether the operator can deal with an unchunked Index more efficiently than a chunked index.

Field Details

ZERO_LENGTH_ITERATIVE_CHUNKED_AGGREGATION_OPERATOR_ARRAY

static final IterativeChunkedAggregationOperator[] ZERO_LENGTH_ITERATIVE_CHUNKED_AGGREGATION_OPERATOR_ARRAY

Method Details

addChunk

void addChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Aggregate a chunk of data into the result columns.

Parameters:

context - the operator-specific context

values - a chunk of values to aggregate

inputIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

removeChunk

void removeChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Remove a chunk of data previously aggregated into the result columns.

Parameters:

context - the operator-specific context

values - a chunk of values that have been previously aggregated.

inputIndices - the input indices, in pre-shift space

destinations - the destinations in resultColumn to remove the values from, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

modifyChunk

default void modifyChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values. Never includes modifies that have been shifted if requiresIndices() returns true - those are handled in shiftChunk(BucketedContext, Chunk, Chunk, LongChunk, LongChunk, IntChunk, IntChunk, IntChunk, WritableBooleanChunk).

Parameters:

context - the operator-specific context

previousValues - a chunk of values that have been previously aggregated

newValues - a chunk of values to aggregate

postShiftIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to remove the values from, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

shiftChunk

default void shiftChunk(IterativeChunkedAggregationOperator.BucketedContext context,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> preShiftIndices,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Called with shifted indices when requiresIndices() returns true, including shifted same-slot modifies.

Parameters:

context - the operator-specific context

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

preShiftIndices - the input indices, in pre-shift space

postShiftIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

modifyIndices

default void modifyIndices(IterativeChunkedAggregationOperator.BucketedContext context,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 IntChunk<Attributes.KeyIndices> destinations,
 IntChunk<Attributes.ChunkPositions> startPositions,
 IntChunk<Attributes.ChunkLengths> length,
 WritableBooleanChunk<Attributes.Values> stateModified)

Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).

Parameters:

context - the operator-specific context

inputIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

addChunk

boolean addChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 int chunkSize,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 long destination)

Aggregate a chunk of data into the result columns.

Parameters:

context - the operator-specific context

chunkSize - the size of the addition

values - the values to aggregate

inputIndices - the input indices, in post-shift space

destination - the destination in the result columns

Returns:

true if the state was modified, false otherwise

removeChunk

boolean removeChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 int chunkSize,
 Chunk<? extends Attributes.Values> values,
 LongChunk<? extends Attributes.KeyIndices> inputIndices,
 long destination)

Remove a chunk of data previously aggregated into the result columns.

Parameters:

context - the operator-specific context

chunkSize - the size of the removal

values - the values to remove from the aggregation

inputIndices - the input indices, in pre-shift space

destination - the destination in the result columns

Returns:

true if the state was modified, false otherwise

modifyChunk

default boolean modifyChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 int chunkSize,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 long destination)

Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values. Never includes modifies that have been shifted if requiresIndices() returns true - those are handled in shiftChunk(SingletonContext, Chunk, Chunk, LongChunk, LongChunk, long).

Parameters:

context - the operator-specific context

chunkSize - the size of the modification

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

postShiftIndices - the input indices, in post-shift space

Returns:

true if the state was modified, false otherwise

shiftChunk

default boolean shiftChunk(IterativeChunkedAggregationOperator.SingletonContext context,
 Chunk<? extends Attributes.Values> previousValues,
 Chunk<? extends Attributes.Values> newValues,
 LongChunk<? extends Attributes.KeyIndices> preShiftIndices,
 LongChunk<? extends Attributes.KeyIndices> postShiftIndices,
 long destination)

Shift a chunk of data previously aggregated into the result columns, including shifted same-slot modifies..

Parameters:

context - the operator-specific context

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

preShiftIndices - the input indices, in pre-shift space

postShiftIndices - the input indices, in post-shift space

destination - the destination in the result columns

Returns:

true if the result should be considered modified

modifyIndices

default boolean modifyIndices(IterativeChunkedAggregationOperator.SingletonContext context,
 LongChunk<? extends Attributes.KeyIndices> indices,
 long destination)

Called with the modified indices when requiresIndices() returns true if our input columns have not changed (or we have none).

Parameters:

context - the operator-specific context

indices - the modified indices for a given destination, in post-shift space

destination - the destination that was modified

Returns:

true if the result should be considered modified

requiresIndices

default boolean requiresIndices()

Whether the operator requires indices. This implies that the operator must process shifts (i.e. shiftChunk(com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator.BucketedContext, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkPositions>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkLengths>, com.illumon.iris.db.v2.sources.chunk.WritableBooleanChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.Values>)), and must observe modifications even when its input columns (if any) are not modified (i.e. modifyIndices(com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator.BucketedContext, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkPositions>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkLengths>, com.illumon.iris.db.v2.sources.chunk.WritableBooleanChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.Values>)).

Returns:

true if the operator requires indices, false otherwise

unchunkedIndex

default boolean unchunkedIndex()

Whether the operator can deal with an unchunked Index more efficiently than a chunked index.

Returns:

true if the operator can deal with unchunked indices, false otherwise

addIndex

default boolean addIndex(IterativeChunkedAggregationOperator.SingletonContext context,
 ReadOnlyIndex index,
 long destination)

ensureCapacity

void ensureCapacity(long tableSize)

Ensure that this operator can handle destinations up to tableSize - 1.

Parameters:

tableSize - the new size of the table

getResultColumns

Map<String,? extends ColumnSource<?>> getResultColumns()

Return a map of result columns produced by this operator.

Returns:

a map of name to columns for the result table

propagateInitialState

default void propagateInitialState(@NotNull
 QueryTable resultTable)

Perform any internal state keeping needed for destinations that were added during initialization.

Parameters:

resultTable - The result QueryTable after initialization

startTrackingPrevValues

void startTrackingPrevValues()

Called after initialization; when the operator's result columns must have previous tracking enabled.

initializeRefreshing

default UnaryOperator<ModifiedColumnSet> initializeRefreshing(@NotNull
 QueryTable resultTable,
 @NotNull
 LivenessReferent aggregationUpdateListener)

Initialize refreshing result support for this operator. As a side effect, make a factory method for converting upstream modified column sets to result modified column sets, to be invoked whenever this operator reports a modification in order to determine the operator's contribution to the final result modified column set.

Parameters:

resultTable - The result QueryTable after initialization

aggregationUpdateListener - The aggregation update listener, which may be needed for referential integrity

Returns:

A factory that produces a result modified column set from the upstream modified column set

resetForStep

default void resetForStep(@NotNull
 ShiftAwareListener.Update upstream)

Reset any per-step internal state. Note that the arguments to this method should not be mutated in any way.

Parameters:

upstream - The upstream ShiftAwareListener.Update

propagateUpdates

default void propagateUpdates(@NotNull
 ShiftAwareListener.Update downstream,
 @NotNull
 ReadOnlyIndex newDestinations)

Perform any internal state keeping needed for destinations that were added (went from 0 keys to > 0), removed (went from > 0 keys to 0), or modified (keys added or removed, or keys modified) by this iteration. Note that the arguments to this method should not be mutated in any way.

Parameters:

downstream - The downstream ShiftAwareListener.Update (which does not have its ModifiedColumnSet finalized yet)

newDestinations - New destinations added on this update

propagateFailure

default void propagateFailure(@NotNull
 Throwable originalException,
 @NotNull
 UpdatePerformanceTracker.Entry sourceEntry)

Called on error to propagate listener failure to this operator.

Parameters:

originalException - The error Throwable

sourceEntry - The UpdatePerformanceTracker.Entry for the failed listener

makeBucketedContext

default IterativeChunkedAggregationOperator.BucketedContext makeBucketedContext(int size)

Make a IterativeChunkedAggregationOperator.BucketedContext suitable for this operator if necessary.

Parameters:

size - The maximum size of input chunks that will be used with the result context

Returns:

A new IterativeChunkedAggregationOperator.BucketedContext, or null if none is necessary

makeSingletonContext

default IterativeChunkedAggregationOperator.SingletonContext makeSingletonContext(int size)

Make a IterativeChunkedAggregationOperator.SingletonContext suitable for this operator if necessary.

Parameters:

size - The maximum size of input chunks that will be used with the result context

Returns:

A new IterativeChunkedAggregationOperator.SingletonContext, or null if none is necessary