Package com.illumon.iris.db.v2.updateby.internal

Class LongRecordingUpdateByOperator

java.lang.Object

com.illumon.iris.db.v2.updateby.internal.LongRecordingUpdateByOperator

All Implemented Interfaces:

UpdateByOperator

public class LongRecordingUpdateByOperator
extends Object
implements UpdateByOperator

An operator that simply remembers the current chunks during the add and reprocess phases.

Nested Class Summary

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

UpdateByOperator.UpdateContext

Field Summary

Fields inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

ZERO_LENGTH_OP_ARRAY

Constructor Summary

Constructors

Constructor	Description
LongRecordingUpdateByOperator(String inputColumnName, String[] affectingColumns, ColumnSource<?> columnSource)

Method Summary

Modifier and Type	Method	Description
void	addChunk(UpdateByOperator.UpdateContext context, Chunk<Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> keyChunk, IntChunk<Attributes.KeyIndices> bucketPositions, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> runLengths)	Add a chunk of bucketed items to the operation.
void	addChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys inputKeys, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> values, long bucketPosition)	Add a chunk of non-bucketed items to the operation.
boolean	anyModified(UpdateByOperator.UpdateContext ctx)	Check if the update has modified any rows for this operator.
void	applyOutputShift(UpdateByOperator.UpdateContext context, ReadOnlyIndex subIndexToShift, long delta)	Apply a shift to the operation.
void	applyShift(UpdateByOperator.UpdateContext updateContext, ReadOnlyIndex prevIndex, IndexShiftData shifted)	Apply a shift to the operation.
boolean	canProcessNormalUpdate(UpdateByOperator.UpdateContext context)	Query if this operator can process the update normally, or if it can only reprocess.
void	finishFor(UpdateByOperator.UpdateContext updateContext, UpdateBy.UpdateType type)	Perform and bookkeeping required at the end of a single part of the update.
ReadOnlyIndex	getAdditionalModifications(UpdateByOperator.UpdateContext ctx)	Get an index of rows that were modified beyond the input set of modifications from the upstream.
String[]	getAffectingColumnNames()	Get an array of column names that, when modified, affect the result of this computation.
long	getCurrentLong(long key)	Get the current long value within the underlying ColumnSource.
String	getInputColumnName()	Get the name of the input column this operator depends on.
long	getLong(int chunkPosition)	Get the long value at the specified position in the currently retained chunk.
String[]	getOutputColumnNames()	Get an array of the output column names.
Map<String,ColumnSource<?>>	getOutputColumns()	Get a map of outputName to output ColumnSource for this operation.
void	initializeFor(UpdateByOperator.UpdateContext updateContext, ReadOnlyIndex updateIndex, UpdateBy.UpdateType type)	Initialize the context for the specified stage of the update process.
void	initializeForUpdate(UpdateByOperator.UpdateContext ctx, ShiftAwareListener.Update upstream, ReadOnlyIndex resultSourceIndex, boolean usingBuckets, boolean isUpstreamAppendOnly)	Initialize the operator for an update cycle.
UpdateByOperator.UpdateContext	makeUpdateContext(int chunkSize)	Make an UpdateByOperator.UpdateContext suitable for use with non-bucketed updates.
void	modifyChunk(UpdateByOperator.UpdateContext updateContext, LongChunk<Attributes.OrderedKeyIndices> prevKeyChunk, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> prevValuesChunk, Chunk<Attributes.Values> postValuesChunk, long bucketPosition)	Modify a chunk of values with the operation.
void	onBucketsRemoved(ReadOnlyIndex removedBuckets)	Called when some buckets have been completely emptied.
void	removeChunk(UpdateByOperator.UpdateContext updateContext, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> prevValuesChunk, long bucketPosition)	Remove a chunk of values from the operation.
void	reprocessChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys chunkOk, Chunk<Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> keyChunk, IntChunk<Attributes.KeyIndices> bucketPositions, IntChunk<Attributes.ChunkPositions> runStartPositions, IntChunk<Attributes.ChunkLengths> runLengths)	Reprocess a chunk of data for a bucketed table.
void	reprocessChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys inputKeys, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> valuesChunk, ReadOnlyIndex postUpdateSourceIndex)	Reprocess a chunk of data for a non-bucketed table.
boolean	requiresKeys()	Query if the operator requires key values for the current stage.
boolean	requiresValues(UpdateByOperator.UpdateContext ctx)	Query if the operator requires values for the current stage.
void	resetForReprocess(UpdateByOperator.UpdateContext context, ReadOnlyIndex sourceIndex, long firstUnmodifiedKey)	Reset the operator to the state at the `firstModifiedKey` for non-bucketed operation.
void	resetForReprocess(UpdateByOperator.UpdateContext context, ReadOnlyIndex bucketIndex, long bucketPosition, long firstUnmodifiedKey)	Reset the operator to the state at the `firstModifiedKey` for the specified bucket.
void	setBucketCapacity(int capacity)	Notify the operator of the current maximum bucket.
void	setChunkSize(UpdateByOperator.UpdateContext context, int chunkSize)	Set the chunk size to be used for operations.
void	startTrackingPrev()	Indicate that the operation should start tracking previous values for ticking updates.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

LongRecordingUpdateByOperator

public LongRecordingUpdateByOperator(@NotNull String inputColumnName, @NotNull String[] affectingColumns, @NotNull ColumnSource<?> columnSource)

Method Details

getLong

public long getLong(int chunkPosition)

Get the long value at the specified position in the currently retained chunk.

Parameters:

chunkPosition - the position

Returns:

the value at the position

getCurrentLong

public long getCurrentLong(long key)

Get the current long value within the underlying ColumnSource.

Parameters:

key - the key

Returns:

the current value at the key within the column source.

reprocessChunk

public void reprocessChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull OrderedKeys inputKeys, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> valuesChunk, @NotNull ReadOnlyIndex postUpdateSourceIndex)

Description copied from interface: UpdateByOperator

Reprocess a chunk of data for a non-bucketed table.

Specified by:

reprocessChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

inputKeys - the keys contained in the chunk

keyChunk - a LongChunk containing the keys if requested by UpdateByOperator.requiresKeys() or null.

valuesChunk - the current chunk of working values.

postUpdateSourceIndex - the resulting source index af

reprocessChunk

public void reprocessChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull OrderedKeys chunkOk, @NotNull Chunk<Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> keyChunk, @NotNull IntChunk<Attributes.KeyIndices> bucketPositions, @NotNull IntChunk<Attributes.ChunkPositions> runStartPositions, @NotNull IntChunk<Attributes.ChunkLengths> runLengths)

Description copied from interface: UpdateByOperator

Reprocess a chunk of data for a bucketed table.

Specified by:

reprocessChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

chunkOk - the keys contained in the chunk

values - the current chunk of working values.

keyChunk - a LongChunk containing the keys.

bucketPositions - a IntChunk containing the bucket position of each key. Parallel to `keyChunk` and `values

runStartPositions - the starting positions of each run within the key and value chunk

runLengths - the run runLengths of each run in the key and value chunks. Parallel to `runStartPositions`

makeUpdateContext

@NotNull public UpdateByOperator.UpdateContext makeUpdateContext(int chunkSize)

Description copied from interface: UpdateByOperator

Make an UpdateByOperator.UpdateContext suitable for use with non-bucketed updates.

Specified by:

makeUpdateContext in interface UpdateByOperator

Parameters:

chunkSize - The expected size of chunks that will be provided during the update,

Returns:

a new context

addChunk

public void addChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull OrderedKeys inputKeys, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> values, long bucketPosition)

Description copied from interface: UpdateByOperator

Add a chunk of non-bucketed items to the operation.

Specified by:

addChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

keyChunk - a chunk of keys for the rows being added. If the operator returns false for UpdateByOperator.requiresKeys() this will be null.

values - the chunk of values for the rows being added

bucketPosition - the group position

addChunk

public void addChunk(@NotNull UpdateByOperator.UpdateContext context, @NotNull Chunk<Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> keyChunk, @NotNull IntChunk<Attributes.KeyIndices> bucketPositions, @NotNull IntChunk<Attributes.ChunkPositions> startPositions, @NotNull IntChunk<Attributes.ChunkLengths> runLengths)

Description copied from interface: UpdateByOperator

Add a chunk of bucketed items to the operation.

Specified by:

addChunk in interface UpdateByOperator

Parameters:

context - the context object

values - the value chunk

keyChunk - a chunk of keys for the rows being added

bucketPositions - a chunk of hash bucket positions for each key

startPositions - the start position of a run within the chunk

runLengths - the runLengths of each run of bucket values

setBucketCapacity

public void setBucketCapacity(int capacity)

Description copied from interface: UpdateByOperator

Notify the operator of the current maximum bucket.

Specified by:

setBucketCapacity in interface UpdateByOperator

Parameters:

capacity - the capacity

getInputColumnName

@NotNull public String getInputColumnName()

Description copied from interface: UpdateByOperator

Get the name of the input column this operator depends on.

Specified by:

getInputColumnName in interface UpdateByOperator

Returns:

the name of the input column

getAffectingColumnNames

@NotNull public String[] getAffectingColumnNames()

Description copied from interface: UpdateByOperator

Get an array of column names that, when modified, affect the result of this computation.

Specified by:

getAffectingColumnNames in interface UpdateByOperator

Returns:

an array of column names that affect this operator.

getOutputColumnNames

@NotNull public String[] getOutputColumnNames()

Description copied from interface: UpdateByOperator

Get an array of the output column names.

Specified by:

getOutputColumnNames in interface UpdateByOperator

Returns:

the output column names.

getOutputColumns

@NotNull public Map<String,ColumnSource<?>> getOutputColumns()

Description copied from interface: UpdateByOperator

Get a map of outputName to output ColumnSource for this operation.

Specified by:

getOutputColumns in interface UpdateByOperator

Returns:

a map of output column name to output column source

initializeForUpdate

public void initializeForUpdate(@NotNull UpdateByOperator.UpdateContext ctx, @NotNull ShiftAwareListener.Update upstream, @NotNull ReadOnlyIndex resultSourceIndex, boolean usingBuckets, boolean isUpstreamAppendOnly)

Description copied from interface: UpdateByOperator

Initialize the operator for an update cycle. This is invoked before any other update processing occurs.

Specified by:

initializeForUpdate in interface UpdateByOperator

Parameters:

ctx - the context object

upstream - the upstream update to process

resultSourceIndex - the result index of the source table

usingBuckets - if the update is bucketed or not

isUpstreamAppendOnly - if the upstream update was detected to be append-only.

initializeFor

public void initializeFor(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull ReadOnlyIndex updateIndex, @NotNull UpdateBy.UpdateType type)

Description copied from interface: UpdateByOperator

Initialize the context for the specified stage of the update process. This will always be followed by a call to UpdateByOperator.finishFor(UpdateContext, UpdateType) at the end of each successful update.

Specified by:

initializeFor in interface UpdateByOperator

Parameters:

updateContext - the context object

updateIndex - the index of rows associated with the update.

type - the type of update being applied

finishFor

public void finishFor(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull UpdateBy.UpdateType type)

Description copied from interface: UpdateByOperator

Perform and bookkeeping required at the end of a single part of the update. This is always preceded with a call to UpdateByOperator.initializeFor(UpdateContext, ReadOnlyIndex, UpdateType)

Specified by:

finishFor in interface UpdateByOperator

Parameters:

updateContext - the context object

type - the type of update being applied

getAdditionalModifications

@NotNull public ReadOnlyIndex getAdditionalModifications(@NotNull UpdateByOperator.UpdateContext ctx)

Description copied from interface: UpdateByOperator

Get an index of rows that were modified beyond the input set of modifications from the upstream. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed) if UpdateByOperator.anyModified(UpdateContext) has returned true.

Specified by:

getAdditionalModifications in interface UpdateByOperator

Parameters:

ctx - the context object

Returns:

a index of additional rows that were modified

anyModified

public boolean anyModified(@NotNull UpdateByOperator.UpdateContext ctx)

Description copied from interface: UpdateByOperator

Check if the update has modified any rows for this operator. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed).

Specified by:

anyModified in interface UpdateByOperator

Parameters:

ctx - the context object

Returns:

true if the update modified any rows.

startTrackingPrev

public void startTrackingPrev()

Description copied from interface: UpdateByOperator

Indicate that the operation should start tracking previous values for ticking updates.

Specified by:

startTrackingPrev in interface UpdateByOperator

requiresKeys

public boolean requiresKeys()

Description copied from interface: UpdateByOperator

Query if the operator requires key values for the current stage. This method will always be invoked after an appropriate invocation of UpdateByOperator.initializeFor(UpdateContext, ReadOnlyIndex, UpdateType)

Specified by:

requiresKeys in interface UpdateByOperator

Returns:

true if the operator requires keys for this operation

requiresValues

public boolean requiresValues(@NotNull UpdateByOperator.UpdateContext ctx)

Description copied from interface: UpdateByOperator

Query if the operator requires values for the current stage.

Specified by:

requiresValues in interface UpdateByOperator

Parameters:

ctx - the context object

Returns:

true if values are required for compuitation

canProcessNormalUpdate

public boolean canProcessNormalUpdate(@NotNull UpdateByOperator.UpdateContext context)

Description copied from interface: UpdateByOperator

Query if this operator can process the update normally, or if it can only reprocess. This method is guaranteed to be invoked after UpdateByOperator.initializeForUpdate(UpdateContext, ShiftAwareListener.Update, ReadOnlyIndex, boolean, boolean) so the operator is aware of the upstream ShiftAwareListener.Update.

Specified by:

canProcessNormalUpdate in interface UpdateByOperator

Parameters:

context - the context

Returns:

true if this operator can process the update normally.

setChunkSize

public void setChunkSize(@NotNull UpdateByOperator.UpdateContext context, int chunkSize)

Description copied from interface: UpdateByOperator

Set the chunk size to be used for operations. This is used during the UpdateBy.UpdateType.Reprocess phase when the chunks allocated during the normal processing phase may not be large enough.

Specified by:

setChunkSize in interface UpdateByOperator

Parameters:

context - the context object

chunkSize - the new chunk size

onBucketsRemoved

public void onBucketsRemoved(@NotNull ReadOnlyIndex removedBuckets)

Description copied from interface: UpdateByOperator

Called when some buckets have been completely emptied. Operators can use this to reset internal states.

Specified by:

onBucketsRemoved in interface UpdateByOperator

Parameters:

removedBuckets - an index of removed bucket positions.

modifyChunk

public void modifyChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @Nullable LongChunk<Attributes.OrderedKeyIndices> prevKeyChunk, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> prevValuesChunk, @NotNull Chunk<Attributes.Values> postValuesChunk, long bucketPosition)

Description copied from interface: UpdateByOperator

Modify a chunk of values with the operation.

Specified by:

modifyChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

prevKeyChunk - a chunk of pre-shift keys. This will be equal to keyChunk if no shift is present

keyChunk - a chunk of post-shift space keys for the update.

prevValuesChunk - a chunk of previous values for the update

postValuesChunk - a chunk of current values for the update

bucketPosition - the position of the current group being processed

removeChunk

public void removeChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> prevValuesChunk, long bucketPosition)

Description copied from interface: UpdateByOperator

Remove a chunk of values from the operation.

Specified by:

removeChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

keyChunk - a chunk of keys being removed.

prevValuesChunk - the chunk of values being removed

bucketPosition - the position of the current group being processed

applyShift

public void applyShift(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull ReadOnlyIndex prevIndex, @NotNull IndexShiftData shifted)

Description copied from interface: UpdateByOperator

Apply a shift to the operation.

Specified by:

applyShift in interface UpdateByOperator

Parameters:

updateContext - the context object

prevIndex - the pre-shifted index

shifted - the shifts being applied

applyOutputShift

public void applyOutputShift(@NotNull UpdateByOperator.UpdateContext context, @NotNull ReadOnlyIndex subIndexToShift, long delta)

Description copied from interface: UpdateByOperator

Apply a shift to the operation.

Specified by:

applyOutputShift in interface UpdateByOperator

Parameters:

context - the context object

resetForReprocess

public void resetForReprocess(@NotNull UpdateByOperator.UpdateContext context, @NotNull ReadOnlyIndex sourceIndex, long firstUnmodifiedKey)

Description copied from interface: UpdateByOperator

Reset the operator to the state at the `firstModifiedKey` for non-bucketed operation. This is invoked immediately prior to calls to UpdateByOperator.reprocessChunk(UpdateContext, OrderedKeys, LongChunk, Chunk, ReadOnlyIndex).

A `firstUnmodifiedKey` of ReadOnlyIndex.NULL_KEY indicates that the entire table needs to be recomputed.

Specified by:

resetForReprocess in interface UpdateByOperator

Parameters:

context - the context object

sourceIndex - the current index of the source table

firstUnmodifiedKey - the first unmodified key after which we will reprocess rows.

resetForReprocess

public void resetForReprocess(@NotNull UpdateByOperator.UpdateContext context, @NotNull ReadOnlyIndex bucketIndex, long bucketPosition, long firstUnmodifiedKey)

Description copied from interface: UpdateByOperator

Reset the operator to the state at the `firstModifiedKey` for the specified bucket. This is invoked immediately prior to calls to UpdateByOperator.reprocessChunk(UpdateContext, OrderedKeys, Chunk, LongChunk, IntChunk, IntChunk, IntChunk).

Specified by:

resetForReprocess in interface UpdateByOperator

Parameters:

context - the context object

bucketIndex - the current index of the specified bucket

firstUnmodifiedKey - the first unmodified key in the bucket after which we will reprocess rows.