Package com.illumon.iris.db.v2.updateby.internal

Class BaseLongUpdateByOperator

java.lang.Object

com.illumon.iris.db.v2.updateby.internal.BaseLongUpdateByOperator

All Implemented Interfaces:

UpdateByOperator

Direct Known Subclasses:

ByteCumProdOperator, ByteCumSumOperator, IntCumProdOperator, IntCumSumOperator, LongCumMinMaxOperator, LongCumProdOperator, LongCumSumOperator, LongDeltaOperator, LongFillByOperator, ShortCumProdOperator, ShortCumSumOperator

public abstract class BaseLongUpdateByOperator
extends Object
implements UpdateByOperator

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	BaseLongUpdateByOperator.Context

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

UpdateByOperator.UpdateContext

Field Summary

Fields

Modifier and Type	Field	Description
protected final String[]	affectingColumns
protected LongArraySource	bucketLastVal
protected boolean	initialized
protected final boolean	isRedirected
protected final WritableSource<Long>	maybeInnerSource
protected final WritableSource<Long>	outputSource
protected final MatchPair	pair
protected long	singletonGroup
protected long	singletonVal	These are only used in grouped operations

Fields inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

ZERO_LENGTH_OP_ARRAY

Constructor Summary

Constructors

Constructor	Description
BaseLongUpdateByOperator(MatchPair pair, String[] affectingColumns, RedirectionIndex redirectionIndex)	Construct a base operator for operations that produce long outputs.

Method Summary

Modifier and Type	Method	Description
void	addChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys inputKeys, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> values, long bucketPosition)	Add a chunk of non-bucketed items to the operation.
final boolean	anyModified(UpdateByOperator.UpdateContext ctx)	Check if the update has modified any rows for this operator.
void	applyOutputShift(UpdateByOperator.UpdateContext context, ReadOnlyIndex subIndexToShift, long delta)	Apply a shift to the operation.
final void	applyShift(UpdateByOperator.UpdateContext updateContext, ReadOnlyIndex prevIndex, IndexShiftData shifted)	Apply a shift to the operation.
boolean	canProcessNormalUpdate(UpdateByOperator.UpdateContext context)	Query if this operator can process the update normally, or if it can only reprocess.
protected abstract void	doAddChunk(BaseLongUpdateByOperator.Context ctx, OrderedKeys inputKeys, Chunk<Attributes.Values> workingChunk, long bucketPosition)	Add a chunk of values to the operator.
void	finishFor(UpdateByOperator.UpdateContext updateContext, UpdateBy.UpdateType type)	Perform and bookkeeping required at the end of a single part of the update.
final ReadOnlyIndex	getAdditionalModifications(UpdateByOperator.UpdateContext ctx)	Get an index of rows that were modified beyond the input set of modifications from the upstream.
String[]	getAffectingColumnNames()	Get an array of column names that, when modified, affect the result of this computation.
String	getInputColumnName()	Get the name of the input column this operator depends on.
String[]	getOutputColumnNames()	Get an array of the output column names.
Map<String,ColumnSource<?>>	getOutputColumns()	Get a map of outputName to output ColumnSource for this operation.
void	initializeFor(UpdateByOperator.UpdateContext updateContext, ReadOnlyIndex updateIndex, UpdateBy.UpdateType type)	Initialize the context for the specified stage of the update process.
void	initializeForUpdate(UpdateByOperator.UpdateContext context, ShiftAwareListener.Update upstream, ReadOnlyIndex resultSourceIndex, boolean usingBuckets, boolean isUpstreamAppendOnly)	Initialize the operator for an update cycle.
UpdateByOperator.UpdateContext	makeUpdateContext(int chunkSize)	Make an UpdateByOperator.UpdateContext suitable for use with non-bucketed updates.
final void	modifyChunk(UpdateByOperator.UpdateContext updateContext, LongChunk<Attributes.OrderedKeyIndices> prevKeyChunk, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> prevValuesChunk, Chunk<Attributes.Values> postValuesChunk, long bucketPosition)	Modify a chunk of values with the operation.
void	onBucketsRemoved(ReadOnlyIndex removedBuckets)	Called when some buckets have been completely emptied.
final void	removeChunk(UpdateByOperator.UpdateContext updateContext, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> prevValuesChunk, long bucketPosition)	Remove a chunk of values from the operation.
void	reprocessChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys chunkOk, Chunk<Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> keyChunk, IntChunk<Attributes.KeyIndices> bucketPositions, IntChunk<Attributes.ChunkPositions> runStartPositions, IntChunk<Attributes.ChunkLengths> runLengths)	Reprocess a chunk of data for a bucketed table.
void	reprocessChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys inputKeys, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> valuesChunk, ReadOnlyIndex postUpdateSourceIndex)	Reprocess a chunk of data for a non-bucketed table.
boolean	requiresKeys()	Query if the operator requires key values for the current stage.
boolean	requiresValues(UpdateByOperator.UpdateContext context)	Query if the operator requires values for the current stage.
void	resetForReprocess(UpdateByOperator.UpdateContext context, ReadOnlyIndex sourceIndex, ColumnSource<?> inputSource, long firstUnmodifiedKey)	Reset the operator to the state at the `firstModifiedKey` for non-bucketed operation.
void	resetForReprocess(UpdateByOperator.UpdateContext ctx, ReadOnlyIndex bucketIndex, ColumnSource<?> inputSource, long bucketPosition, long firstUnmodifiedKey)	Reset the operator to the state at the `firstModifiedKey` for the specified bucket.
void	setBucketCapacity(int capacity)	Notify the operator of the current maximum bucket.
void	setChunkSize(UpdateByOperator.UpdateContext context, int chunkSize)	Set the chunk size to be used for operations.
void	startTrackingPrev()	Indicate that the operation should start tracking previous values for ticking updates.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

addChunk

Field Details

outputSource

protected final WritableSource<Long> outputSource

maybeInnerSource

protected final WritableSource<Long> maybeInnerSource

pair

protected final MatchPair pair

affectingColumns

protected final String[] affectingColumns

isRedirected

protected final boolean isRedirected

bucketLastVal

protected LongArraySource bucketLastVal

singletonVal

protected long singletonVal

These are only used in grouped operations

singletonGroup

protected long singletonGroup

initialized

protected boolean initialized

Constructor Details

BaseLongUpdateByOperator

public BaseLongUpdateByOperator(@NotNull
 MatchPair pair,
 @NotNull
 String[] affectingColumns,
 @Nullable
 RedirectionIndex redirectionIndex)

Construct a base operator for operations that produce long outputs.

Parameters:

pair - the MatchPair that defines the input/output for this operation

affectingColumns - a list of all columns (including the input column from the pair) that affects the result of this operator.

redirectionIndex - the RedirectionIndex if one is used

Method Details

setChunkSize

public void setChunkSize(@NotNull
 UpdateByOperator.UpdateContext context,
 int chunkSize)

Description copied from interface: UpdateByOperator

Set the chunk size to be used for operations. This is used during the UpdateBy.UpdateType.Reprocess phase when the chunks allocated during the normal processing phase may not be large enough.

Specified by:

setChunkSize in interface UpdateByOperator

Parameters:

context - the context object

chunkSize - the new chunk size

setBucketCapacity

public void setBucketCapacity(int capacity)

Description copied from interface: UpdateByOperator

Notify the operator of the current maximum bucket.

Specified by:

setBucketCapacity in interface UpdateByOperator

Parameters:

capacity - the capacity

getInputColumnName

@NotNull
public String getInputColumnName()

Description copied from interface: UpdateByOperator

Get the name of the input column this operator depends on.

Specified by:

getInputColumnName in interface UpdateByOperator

Returns:

the name of the input column

getAffectingColumnNames

@NotNull
public String[] getAffectingColumnNames()

Description copied from interface: UpdateByOperator

Get an array of column names that, when modified, affect the result of this computation.

Specified by:

getAffectingColumnNames in interface UpdateByOperator

Returns:

an array of column names that affect this operator.

getOutputColumnNames

@NotNull
public String[] getOutputColumnNames()

Description copied from interface: UpdateByOperator

Get an array of the output column names.

Specified by:

getOutputColumnNames in interface UpdateByOperator

Returns:

the output column names.

getOutputColumns

@NotNull
public Map<String,ColumnSource<?>> getOutputColumns()

Description copied from interface: UpdateByOperator

Get a map of outputName to output ColumnSource for this operation.

Specified by:

getOutputColumns in interface UpdateByOperator

Returns:

a map of output column name to output column source

makeUpdateContext

@NotNull
public UpdateByOperator.UpdateContext makeUpdateContext(int chunkSize)

Description copied from interface: UpdateByOperator

Make an UpdateByOperator.UpdateContext suitable for use with non-bucketed updates.

Specified by:

makeUpdateContext in interface UpdateByOperator

Parameters:

chunkSize - The expected size of chunks that will be provided during the update,

Returns:

a new context

initializeForUpdate

public void initializeForUpdate(@NotNull
 UpdateByOperator.UpdateContext context,
 @NotNull
 ShiftAwareListener.Update upstream,
 @NotNull
 ReadOnlyIndex resultSourceIndex,
 boolean usingBuckets,
 boolean isUpstreamAppendOnly)

Description copied from interface: UpdateByOperator

Initialize the operator for an update cycle. This is invoked before any other update processing occurs.

Specified by:

initializeForUpdate in interface UpdateByOperator

Parameters:

context - the context object

upstream - the upstream update to process

resultSourceIndex - the result index of the source table

usingBuckets - if the update is bucketed or not

isUpstreamAppendOnly - if the upstream update was detected to be append-only.

initializeFor

public void initializeFor(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @NotNull
 ReadOnlyIndex updateIndex,
 @NotNull
 UpdateBy.UpdateType type)

Description copied from interface: UpdateByOperator

Initialize the context for the specified stage of the update process. This will always be followed by a call to UpdateByOperator.finishFor(UpdateContext, UpdateType) at the end of each successful update.

Specified by:

initializeFor in interface UpdateByOperator

Parameters:

updateContext - the context object

updateIndex - the index of rows associated with the update.

type - the type of update being applied

finishFor

public void finishFor(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @NotNull
 UpdateBy.UpdateType type)

Description copied from interface: UpdateByOperator

Perform and bookkeeping required at the end of a single part of the update. This is always preceded with a call to UpdateByOperator.initializeFor(UpdateContext, ReadOnlyIndex, UpdateType)

Specified by:

finishFor in interface UpdateByOperator

Parameters:

updateContext - the context object

type - the type of update being applied

requiresKeys

public boolean requiresKeys()

Description copied from interface: UpdateByOperator

Query if the operator requires key values for the current stage. This method will always be invoked after an appropriate invocation of UpdateByOperator.initializeFor(UpdateContext, ReadOnlyIndex, UpdateType)

Specified by:

requiresKeys in interface UpdateByOperator

Returns:

true if the operator requires keys for this operation

requiresValues

public boolean requiresValues(@NotNull
 UpdateByOperator.UpdateContext context)

Description copied from interface: UpdateByOperator

Query if the operator requires values for the current stage.

Specified by:

requiresValues in interface UpdateByOperator

Parameters:

context - the context object

Returns:

true if values are required for compuitation

startTrackingPrev

public void startTrackingPrev()

Description copied from interface: UpdateByOperator

Indicate that the operation should start tracking previous values for ticking updates.

Specified by:

startTrackingPrev in interface UpdateByOperator

getAdditionalModifications

@NotNull
public final ReadOnlyIndex getAdditionalModifications(@NotNull
 UpdateByOperator.UpdateContext ctx)

Description copied from interface: UpdateByOperator

Get an index of rows that were modified beyond the input set of modifications from the upstream. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed) if UpdateByOperator.anyModified(UpdateContext) has returned true.

Specified by:

getAdditionalModifications in interface UpdateByOperator

Parameters:

ctx - the context object

Returns:

a index of additional rows that were modified

anyModified

public final boolean anyModified(@NotNull
 UpdateByOperator.UpdateContext ctx)

Description copied from interface: UpdateByOperator

Check if the update has modified any rows for this operator. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed).

Specified by:

anyModified in interface UpdateByOperator

Parameters:

ctx - the context object

Returns:

true if the update modified any rows.

onBucketsRemoved

public void onBucketsRemoved(@NotNull
 ReadOnlyIndex removedBuckets)

Description copied from interface: UpdateByOperator

Called when some buckets have been completely emptied. Operators can use this to reset internal states.

Specified by:

onBucketsRemoved in interface UpdateByOperator

Parameters:

removedBuckets - an index of removed bucket positions.

canProcessNormalUpdate

public boolean canProcessNormalUpdate(@NotNull
 UpdateByOperator.UpdateContext context)

Description copied from interface: UpdateByOperator

Query if this operator can process the update normally, or if it can only reprocess. This method is guaranteed to be invoked after UpdateByOperator.initializeForUpdate(UpdateContext, ShiftAwareListener.Update, ReadOnlyIndex, boolean, boolean) so the operator is aware of the upstream ShiftAwareListener.Update.

Specified by:

canProcessNormalUpdate in interface UpdateByOperator

Parameters:

context - the context

Returns:

true if this operator can process the update normally.

addChunk

public void addChunk(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @NotNull
 OrderedKeys inputKeys,
 @Nullable
 LongChunk<Attributes.OrderedKeyIndices> keyChunk,
 @NotNull
 Chunk<Attributes.Values> values,
 long bucketPosition)

Description copied from interface: UpdateByOperator

Add a chunk of non-bucketed items to the operation.

Specified by:

addChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

keyChunk - a chunk of keys for the rows being added. If the operator returns false for UpdateByOperator.requiresKeys() this will be null.

values - the chunk of values for the rows being added

bucketPosition - the group position

doAddChunk

protected abstract void doAddChunk(@NotNull
 BaseLongUpdateByOperator.Context ctx,
 @NotNull
 OrderedKeys inputKeys,
 @NotNull
 Chunk<Attributes.Values> workingChunk,
 long bucketPosition)

Add a chunk of values to the operator.

Parameters:

ctx - the context object

inputKeys - the input keys for the chunk

workingChunk - the chunk of values

bucketPosition - the bucket position that the values belong to.

applyOutputShift

public void applyOutputShift(@NotNull
 UpdateByOperator.UpdateContext context,
 @NotNull
 ReadOnlyIndex subIndexToShift,
 long delta)

Description copied from interface: UpdateByOperator

Apply a shift to the operation.

Specified by:

applyOutputShift in interface UpdateByOperator

Parameters:

context - the context object

resetForReprocess

public void resetForReprocess(@NotNull
 UpdateByOperator.UpdateContext context,
 @NotNull
 ReadOnlyIndex sourceIndex,
 @NotNull
 ColumnSource<?> inputSource,
 long firstUnmodifiedKey)

Description copied from interface: UpdateByOperator

Reset the operator to the state at the `firstModifiedKey` for non-bucketed operation. This is invoked immediately prior to calls to UpdateByOperator.reprocessChunk(UpdateContext, OrderedKeys, LongChunk, Chunk, ReadOnlyIndex).

A `firstUnmodifiedKey` of ReadOnlyIndex.NULL_KEY indicates that the entire table needs to be recomputed.

Specified by:

resetForReprocess in interface UpdateByOperator

Parameters:

context - the context object

sourceIndex - the current index of the source table

inputSource - the most primitive form of the input column source

firstUnmodifiedKey - the first unmodified key after which we will reprocess rows.

resetForReprocess

public void resetForReprocess(@NotNull
 UpdateByOperator.UpdateContext ctx,
 @NotNull
 ReadOnlyIndex bucketIndex,
 @NotNull
 ColumnSource<?> inputSource,
 long bucketPosition,
 long firstUnmodifiedKey)

Description copied from interface: UpdateByOperator

Reset the operator to the state at the `firstModifiedKey` for the specified bucket. This is invoked immediately prior to calls to UpdateByOperator.reprocessChunk(UpdateContext, OrderedKeys, Chunk, LongChunk, IntChunk, IntChunk, IntChunk).

Specified by:

resetForReprocess in interface UpdateByOperator

Parameters:

ctx - the context object

bucketIndex - the current index of the specified bucket

inputSource - the most primitive form of the input column source

firstUnmodifiedKey - the first unmodified key in the bucket after which we will reprocess rows.

reprocessChunk

public void reprocessChunk(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @NotNull
 OrderedKeys inputKeys,
 @Nullable
 LongChunk<Attributes.OrderedKeyIndices> keyChunk,
 @NotNull
 Chunk<Attributes.Values> valuesChunk,
 @NotNull
 ReadOnlyIndex postUpdateSourceIndex)

Description copied from interface: UpdateByOperator

Reprocess a chunk of data for a non-bucketed table.

Specified by:

reprocessChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

inputKeys - the keys contained in the chunk

keyChunk - a LongChunk containing the keys if requested by UpdateByOperator.requiresKeys() or null.

valuesChunk - the current chunk of working values.

postUpdateSourceIndex - the resulting source index af

reprocessChunk

public void reprocessChunk(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @NotNull
 OrderedKeys chunkOk,
 @NotNull
 Chunk<Attributes.Values> values,
 @NotNull
 LongChunk<? extends Attributes.KeyIndices> keyChunk,
 @NotNull
 IntChunk<Attributes.KeyIndices> bucketPositions,
 @NotNull
 IntChunk<Attributes.ChunkPositions> runStartPositions,
 @NotNull
 IntChunk<Attributes.ChunkLengths> runLengths)

Description copied from interface: UpdateByOperator

Reprocess a chunk of data for a bucketed table.

Specified by:

reprocessChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

chunkOk - the keys contained in the chunk

values - the current chunk of working values.

keyChunk - a LongChunk containing the keys.

bucketPositions - a IntChunk containing the bucket position of each key. Parallel to `keyChunk` and `values

runStartPositions - the starting positions of each run within the key and value chunk

runLengths - the run runLengths of each run in the key and value chunks. Parallel to `runStartPositions`

modifyChunk

public final void modifyChunk(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @Nullable
 LongChunk<Attributes.OrderedKeyIndices> prevKeyChunk,
 @Nullable
 LongChunk<Attributes.OrderedKeyIndices> keyChunk,
 @NotNull
 Chunk<Attributes.Values> prevValuesChunk,
 @NotNull
 Chunk<Attributes.Values> postValuesChunk,
 long bucketPosition)

Description copied from interface: UpdateByOperator

Modify a chunk of values with the operation.

Specified by:

modifyChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

prevKeyChunk - a chunk of pre-shift keys. This will be equal to keyChunk if no shift is present

keyChunk - a chunk of post-shift space keys for the update.

prevValuesChunk - a chunk of previous values for the update

postValuesChunk - a chunk of current values for the update

bucketPosition - the position of the current group being processed

removeChunk

public final void removeChunk(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @Nullable
 LongChunk<Attributes.OrderedKeyIndices> keyChunk,
 @NotNull
 Chunk<Attributes.Values> prevValuesChunk,
 long bucketPosition)

Description copied from interface: UpdateByOperator

Remove a chunk of values from the operation.

Specified by:

removeChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

keyChunk - a chunk of keys being removed.

prevValuesChunk - the chunk of values being removed

bucketPosition - the position of the current group being processed

applyShift

public final void applyShift(@NotNull
 UpdateByOperator.UpdateContext updateContext,
 @NotNull
 ReadOnlyIndex prevIndex,
 @NotNull
 IndexShiftData shifted)

Description copied from interface: UpdateByOperator

Apply a shift to the operation.

Specified by:

applyShift in interface UpdateByOperator

Parameters:

updateContext - the context object

prevIndex - the pre-shifted index

shifted - the shifts being applied