Package com.illumon.iris.db.v2.by

Class ByExternalChunkedOperator

java.lang.Object

com.illumon.iris.db.v2.by.ByExternalChunkedOperator

All Implemented Interfaces:

IterativeChunkedAggregationOperator

public final class ByExternalChunkedOperator
extends Object
implements IterativeChunkedAggregationOperator

An IterativeChunkedAggregationOperator used in the implementation of Table.byExternal(boolean, java.lang.String...).

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	ByExternalChunkedOperator.AttributeCopier

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator

IterativeChunkedAggregationOperator.BucketedContext, IterativeChunkedAggregationOperator.SingletonContext

Field Summary

Fields inherited from interface com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator

ZERO_LENGTH_ITERATIVE_CHUNKED_AGGREGATION_OPERATOR_ARRAY

Method Summary

Modifier and Type	Method	Description
void	addChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Aggregate a chunk of data into the result columns.
boolean	addChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, int chunkSize, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)	Aggregate a chunk of data into the result columns.
boolean	addIndex(IterativeChunkedAggregationOperator.SingletonContext context, ReadOnlyIndex index, long destination)
void	ensureCapacity(long tableSize)	Ensure that this operator can handle destinations up to tableSize - 1.
Map<String,? extends ColumnSource<?>>	getResultColumns()	Return a map of result columns produced by this operator.
UnaryOperator<ModifiedColumnSet>	initializeRefreshing(QueryTable resultTable, LivenessReferent aggregationUpdateListener)	Initialize refreshing result support for this operator.
void	modifyChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values.
boolean	modifyChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, int chunkSize, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, long destination)	Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values.
void	modifyIndices(IterativeChunkedAggregationOperator.BucketedContext context, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Called with the modified indices when IterativeChunkedAggregationOperator.requiresIndices() returns true if our input columns have not changed (or we have none).
boolean	modifyIndices(IterativeChunkedAggregationOperator.SingletonContext context, LongChunk<? extends Attributes.KeyIndices> indices, long destination)	Called with the modified indices when IterativeChunkedAggregationOperator.requiresIndices() returns true if our input columns have not changed (or we have none).
void	propagateFailure(Throwable originalException, UpdatePerformanceTracker.Entry sourceEntry)	Called on error to propagate listener failure to this operator.
void	propagateInitialState(QueryTable resultTable)	Perform any internal state keeping needed for destinations that were added during initialization.
void	propagateUpdates(ShiftAwareListener.Update downstream, ReadOnlyIndex newDestinations)	Perform any internal state keeping needed for destinations that were added (went from 0 keys to > 0), removed (went from > 0 keys to 0), or modified (keys added or removed, or keys modified) by this iteration.
void	removeChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Remove a chunk of data previously aggregated into the result columns.
boolean	removeChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, int chunkSize, Chunk<? extends Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)	Remove a chunk of data previously aggregated into the result columns.
boolean	requiresIndices()	Whether the operator requires indices.
void	resetForStep(ShiftAwareListener.Update upstream)	Reset any per-step internal state.
void	shiftChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> preShiftIndices, LongChunk<? extends Attributes.KeyIndices> postShiftIndices, IntChunk<Attributes.KeyIndices> destinations, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length, WritableBooleanChunk<Attributes.Values> stateModified)	Called with shifted indices when IterativeChunkedAggregationOperator.requiresIndices() returns true, including shifted same-slot modifies.
boolean	shiftChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, LongChunk<? extends Attributes.KeyIndices> preInputIndices, LongChunk<? extends Attributes.KeyIndices> postInputIndices, long destination)	Shift a chunk of data previously aggregated into the result columns, including shifted same-slot modifies..
void	startTrackingPrevValues()	Called after initialization; when the operator's result columns must have previous tracking enabled.
boolean	unchunkedIndex()	Whether the operator can deal with an unchunked Index more efficiently than a chunked index.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator

makeBucketedContext, makeSingletonContext

Method Details

addChunk

public void addChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> inputIndices, @NotNull IntChunk<Attributes.KeyIndices> destinations, @NotNull IntChunk<Attributes.ChunkPositions> startPositions, @NotNull IntChunk<Attributes.ChunkLengths> length, @NotNull WritableBooleanChunk<Attributes.Values> stateModified)

Description copied from interface: IterativeChunkedAggregationOperator

Aggregate a chunk of data into the result columns.

Specified by:

addChunk in interface IterativeChunkedAggregationOperator

Parameters:

bucketedContext - the operator-specific context

values - a chunk of values to aggregate

inputIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

removeChunk

public void removeChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> inputIndices, @NotNull IntChunk<Attributes.KeyIndices> destinations, @NotNull IntChunk<Attributes.ChunkPositions> startPositions, @NotNull IntChunk<Attributes.ChunkLengths> length, @NotNull WritableBooleanChunk<Attributes.Values> stateModified)

Description copied from interface: IterativeChunkedAggregationOperator

Remove a chunk of data previously aggregated into the result columns.

Specified by:

removeChunk in interface IterativeChunkedAggregationOperator

Parameters:

bucketedContext - the operator-specific context

values - a chunk of values that have been previously aggregated.

inputIndices - the input indices, in pre-shift space

destinations - the destinations in resultColumn to remove the values from, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

modifyChunk

public void modifyChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, @NotNull LongChunk<? extends Attributes.KeyIndices> postShiftIndices, @NotNull IntChunk<Attributes.KeyIndices> destinations, @NotNull IntChunk<Attributes.ChunkPositions> startPositions, @NotNull IntChunk<Attributes.ChunkLengths> length, @NotNull WritableBooleanChunk<Attributes.Values> stateModified)

Description copied from interface: IterativeChunkedAggregationOperator

Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values. Never includes modifies that have been shifted if IterativeChunkedAggregationOperator.requiresIndices() returns true - those are handled in IterativeChunkedAggregationOperator.shiftChunk(BucketedContext, Chunk, Chunk, LongChunk, LongChunk, IntChunk, IntChunk, IntChunk, WritableBooleanChunk).

Specified by:

modifyChunk in interface IterativeChunkedAggregationOperator

Parameters:

bucketedContext - the operator-specific context

previousValues - a chunk of values that have been previously aggregated

newValues - a chunk of values to aggregate

postShiftIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to remove the values from, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

shiftChunk

public void shiftChunk(IterativeChunkedAggregationOperator.BucketedContext bucketedContext, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, @NotNull LongChunk<? extends Attributes.KeyIndices> preShiftIndices, @NotNull LongChunk<? extends Attributes.KeyIndices> postShiftIndices, @NotNull IntChunk<Attributes.KeyIndices> destinations, @NotNull IntChunk<Attributes.ChunkPositions> startPositions, @NotNull IntChunk<Attributes.ChunkLengths> length, @NotNull WritableBooleanChunk<Attributes.Values> stateModified)

Description copied from interface: IterativeChunkedAggregationOperator

Called with shifted indices when IterativeChunkedAggregationOperator.requiresIndices() returns true, including shifted same-slot modifies.

Specified by:

shiftChunk in interface IterativeChunkedAggregationOperator

Parameters:

bucketedContext - the operator-specific context

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

preShiftIndices - the input indices, in pre-shift space

postShiftIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

modifyIndices

public void modifyIndices(IterativeChunkedAggregationOperator.BucketedContext context, @NotNull LongChunk<? extends Attributes.KeyIndices> inputIndices, @NotNull IntChunk<Attributes.KeyIndices> destinations, @NotNull IntChunk<Attributes.ChunkPositions> startPositions, @NotNull IntChunk<Attributes.ChunkLengths> length, @NotNull WritableBooleanChunk<Attributes.Values> stateModified)

Description copied from interface: IterativeChunkedAggregationOperator

Called with the modified indices when IterativeChunkedAggregationOperator.requiresIndices() returns true if our input columns have not changed (or we have none).

Specified by:

modifyIndices in interface IterativeChunkedAggregationOperator

Parameters:

context - the operator-specific context

inputIndices - the input indices, in post-shift space

destinations - the destinations in resultColumn to aggregate into, parallel with startPositions and length

startPositions - the starting positions in the chunk for each destination

length - the number of values in the chunk for each destination

stateModified - a boolean output array, parallel to destinations, which is set to true if the corresponding destination has been modified

addChunk

public boolean addChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, int chunkSize, Chunk<? extends Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)

Description copied from interface: IterativeChunkedAggregationOperator

Aggregate a chunk of data into the result columns.

Specified by:

addChunk in interface IterativeChunkedAggregationOperator

Parameters:

singletonContext - the operator-specific context

chunkSize - the size of the addition

values - the values to aggregate

inputIndices - the input indices, in post-shift space

destination - the destination in the result columns

Returns:

true if the state was modified, false otherwise

addIndex

public boolean addIndex(IterativeChunkedAggregationOperator.SingletonContext context, ReadOnlyIndex index, long destination)

Specified by:

addIndex in interface IterativeChunkedAggregationOperator

removeChunk

public boolean removeChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, int chunkSize, Chunk<? extends Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> inputIndices, long destination)

Description copied from interface: IterativeChunkedAggregationOperator

Remove a chunk of data previously aggregated into the result columns.

Specified by:

removeChunk in interface IterativeChunkedAggregationOperator

Parameters:

singletonContext - the operator-specific context

chunkSize - the size of the removal

values - the values to remove from the aggregation

inputIndices - the input indices, in pre-shift space

destination - the destination in the result columns

Returns:

true if the state was modified, false otherwise

modifyChunk

public boolean modifyChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, int chunkSize, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, @NotNull LongChunk<? extends Attributes.KeyIndices> postShiftIndices, long destination)

Description copied from interface: IterativeChunkedAggregationOperator

Modify a chunk of data previously aggregated into the result columns using a parallel chunk of new values. Never includes modifies that have been shifted if IterativeChunkedAggregationOperator.requiresIndices() returns true - those are handled in IterativeChunkedAggregationOperator.shiftChunk(SingletonContext, Chunk, Chunk, LongChunk, LongChunk, long).

Specified by:

modifyChunk in interface IterativeChunkedAggregationOperator

Parameters:

singletonContext - the operator-specific context

chunkSize - the size of the modification

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

postShiftIndices - the input indices, in post-shift space

Returns:

true if the state was modified, false otherwise

shiftChunk

public boolean shiftChunk(IterativeChunkedAggregationOperator.SingletonContext singletonContext, Chunk<? extends Attributes.Values> previousValues, Chunk<? extends Attributes.Values> newValues, @NotNull LongChunk<? extends Attributes.KeyIndices> preInputIndices, @NotNull LongChunk<? extends Attributes.KeyIndices> postInputIndices, long destination)

Description copied from interface: IterativeChunkedAggregationOperator

Shift a chunk of data previously aggregated into the result columns, including shifted same-slot modifies..

Specified by:

shiftChunk in interface IterativeChunkedAggregationOperator

Parameters:

singletonContext - the operator-specific context

previousValues - a chunk of values that have been previously aggregated.

newValues - a chunk of values to aggregate

preInputIndices - the input indices, in pre-shift space

postInputIndices - the input indices, in post-shift space

destination - the destination in the result columns

Returns:

true if the result should be considered modified

modifyIndices

public boolean modifyIndices(IterativeChunkedAggregationOperator.SingletonContext context, @NotNull LongChunk<? extends Attributes.KeyIndices> indices, long destination)

Description copied from interface: IterativeChunkedAggregationOperator

Called with the modified indices when IterativeChunkedAggregationOperator.requiresIndices() returns true if our input columns have not changed (or we have none).

Specified by:

modifyIndices in interface IterativeChunkedAggregationOperator

Parameters:

context - the operator-specific context

indices - the modified indices for a given destination, in post-shift space

destination - the destination that was modified

Returns:

true if the result should be considered modified

ensureCapacity

public void ensureCapacity(long tableSize)

Description copied from interface: IterativeChunkedAggregationOperator

Ensure that this operator can handle destinations up to tableSize - 1.

Specified by:

ensureCapacity in interface IterativeChunkedAggregationOperator

Parameters:

tableSize - the new size of the table

getResultColumns

public Map<String,? extends ColumnSource<?>> getResultColumns()

Description copied from interface: IterativeChunkedAggregationOperator

Return a map of result columns produced by this operator.

Specified by:

getResultColumns in interface IterativeChunkedAggregationOperator

Returns:

a map of name to columns for the result table

startTrackingPrevValues

public void startTrackingPrevValues()

Description copied from interface: IterativeChunkedAggregationOperator

Called after initialization; when the operator's result columns must have previous tracking enabled.

Specified by:

startTrackingPrevValues in interface IterativeChunkedAggregationOperator

propagateInitialState

public void propagateInitialState(@NotNull QueryTable resultTable)

Description copied from interface: IterativeChunkedAggregationOperator

Perform any internal state keeping needed for destinations that were added during initialization.

Specified by:

propagateInitialState in interface IterativeChunkedAggregationOperator

Parameters:

resultTable - The result QueryTable after initialization

initializeRefreshing

public UnaryOperator<ModifiedColumnSet> initializeRefreshing(@NotNull QueryTable resultTable, @NotNull LivenessReferent aggregationUpdateListener)

Description copied from interface: IterativeChunkedAggregationOperator

Initialize refreshing result support for this operator. As a side effect, make a factory method for converting upstream modified column sets to result modified column sets, to be invoked whenever this operator reports a modification in order to determine the operator's contribution to the final result modified column set.

Specified by:

initializeRefreshing in interface IterativeChunkedAggregationOperator

Parameters:

resultTable - The result QueryTable after initialization

aggregationUpdateListener - The aggregation update listener, which may be needed for referential integrity

Returns:

A factory that produces a result modified column set from the upstream modified column set

resetForStep

public void resetForStep(@NotNull ShiftAwareListener.Update upstream)

Description copied from interface: IterativeChunkedAggregationOperator

Reset any per-step internal state. Note that the arguments to this method should not be mutated in any way.

Specified by:

resetForStep in interface IterativeChunkedAggregationOperator

Parameters:

upstream - The upstream ShiftAwareListener.Update

propagateUpdates

public void propagateUpdates(@NotNull ShiftAwareListener.Update downstream, @NotNull ReadOnlyIndex newDestinations)

Description copied from interface: IterativeChunkedAggregationOperator

Perform any internal state keeping needed for destinations that were added (went from 0 keys to > 0), removed (went from > 0 keys to 0), or modified (keys added or removed, or keys modified) by this iteration. Note that the arguments to this method should not be mutated in any way.

Specified by:

propagateUpdates in interface IterativeChunkedAggregationOperator

Parameters:

downstream - The downstream ShiftAwareListener.Update (which does not have its ModifiedColumnSet finalized yet)

newDestinations - New destinations added on this update

propagateFailure

public void propagateFailure(@NotNull Throwable originalException, @NotNull UpdatePerformanceTracker.Entry sourceEntry)

Description copied from interface: IterativeChunkedAggregationOperator

Called on error to propagate listener failure to this operator.

Specified by:

propagateFailure in interface IterativeChunkedAggregationOperator

Parameters:

originalException - The error Throwable

sourceEntry - The UpdatePerformanceTracker.Entry for the failed listener

requiresIndices

public boolean requiresIndices()

Description copied from interface: IterativeChunkedAggregationOperator

Whether the operator requires indices. This implies that the operator must process shifts (i.e. IterativeChunkedAggregationOperator.shiftChunk(com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator.BucketedContext, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkPositions>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkLengths>, com.illumon.iris.db.v2.sources.chunk.WritableBooleanChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.Values>)), and must observe modifications even when its input columns (if any) are not modified (i.e. IterativeChunkedAggregationOperator.modifyIndices(com.illumon.iris.db.v2.by.IterativeChunkedAggregationOperator.BucketedContext, com.illumon.iris.db.v2.sources.chunk.LongChunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.KeyIndices>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkPositions>, com.illumon.iris.db.v2.sources.chunk.IntChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.ChunkLengths>, com.illumon.iris.db.v2.sources.chunk.WritableBooleanChunk<com.illumon.iris.db.v2.sources.chunk.Attributes.Values>)).

Specified by:

requiresIndices in interface IterativeChunkedAggregationOperator

Returns:

true if the operator requires indices, false otherwise

unchunkedIndex

public boolean unchunkedIndex()

Description copied from interface: IterativeChunkedAggregationOperator

Whether the operator can deal with an unchunked Index more efficiently than a chunked index.

Specified by:

unchunkedIndex in interface IterativeChunkedAggregationOperator

Returns:

true if the operator can deal with unchunked indices, false otherwise