Package com.illumon.iris.db.v2.updateby.internal

Class BaseByteUpdateByOperator

java.lang.Object

com.illumon.iris.db.v2.updateby.internal.BaseByteUpdateByOperator

All Implemented Interfaces:

UpdateByOperator

Direct Known Subclasses:

BooleanFillByOperator, ByteCumMinMaxOperator, ByteFillByOperator

public abstract class BaseByteUpdateByOperator
extends Object
implements UpdateByOperator

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	BaseByteUpdateByOperator.Context

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

UpdateByOperator.UpdateContext

Field Summary

Fields

Modifier and Type	Field	Description
protected String[]	affectingColumns
protected WritableSource<Byte>	bucketLastVal
protected boolean	initialized
protected boolean	isRedirected
protected WritableSource<Byte>	maybeInnerSource
protected WritableSource<Byte>	outputSource
protected MatchPair	pair
protected long	singletonGroup
protected byte	singletonVal	These are only used in grouped operations

Fields inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

ZERO_LENGTH_OP_ARRAY

Constructor Summary

Constructors

Constructor	Description
BaseByteUpdateByOperator(MatchPair pair, String[] affectingColumns, RedirectionIndex redirectionIndex)	Construct a base operator for operations that produce byte outputs.

Method Summary

Modifier and Type	Method	Description
void	addChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys inputKeys, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> values, long bucketPosition)	Add a chunk of non-bucketed items to the operation.
boolean	anyModified(UpdateByOperator.UpdateContext ctx)	Check if the update has modified any rows for this operator.
void	applyOutputShift(UpdateByOperator.UpdateContext context, ReadOnlyIndex subIndexToShift, long delta)	Apply a shift to the operation.
void	applyShift(UpdateByOperator.UpdateContext updateContext, ReadOnlyIndex prevIndex, IndexShiftData shifted)	Apply a shift to the operation.
boolean	canProcessNormalUpdate(UpdateByOperator.UpdateContext context)	Query if this operator can process the update normally, or if it can only reprocess.
protected abstract void	doAddChunk(BaseByteUpdateByOperator.Context ctx, OrderedKeys inputKeys, Chunk<Attributes.Values> workingChunk, long bucketPosition)	Add a chunk of values to the operator.
void	finishFor(UpdateByOperator.UpdateContext updateContext, UpdateBy.UpdateType type)	Perform and bookkeeping required at the end of a single part of the update.
ReadOnlyIndex	getAdditionalModifications(UpdateByOperator.UpdateContext ctx)	Get an index of rows that were modified beyond the input set of modifications from the upstream.
String[]	getAffectingColumnNames()	Get an array of column names that, when modified, affect the result of this computation.
String	getInputColumnName()	Get the name of the input column this operator depends on.
protected byte	getNullValue()
String[]	getOutputColumnNames()	Get an array of the output column names.
Map<String,ColumnSource<?>>	getOutputColumns()	Get a map of outputName to output ColumnSource for this operation.
void	initializeFor(UpdateByOperator.UpdateContext updateContext, ReadOnlyIndex updateIndex, UpdateBy.UpdateType type)	Initialize the context for the specified stage of the update process.
void	initializeForUpdate(UpdateByOperator.UpdateContext context, ShiftAwareListener.Update upstream, ReadOnlyIndex resultSourceIndex, boolean usingBuckets, boolean isUpstreamAppendOnly)	Initialize the operator for an update cycle.
protected WritableSource<Byte>	makeDenseSource()
protected WritableSource<Byte>	makeSparseSource()
UpdateByOperator.UpdateContext	makeUpdateContext(int chunkSize)	Make an UpdateByOperator.UpdateContext suitable for use with non-bucketed updates.
void	modifyChunk(UpdateByOperator.UpdateContext updateContext, LongChunk<Attributes.OrderedKeyIndices> prevKeyChunk, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> prevValuesChunk, Chunk<Attributes.Values> postValuesChunk, long bucketPosition)	Modify a chunk of values with the operation.
void	onBucketsRemoved(ReadOnlyIndex removedBuckets)	Called when some buckets have been completely emptied.
void	removeChunk(UpdateByOperator.UpdateContext updateContext, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> prevValuesChunk, long bucketPosition)	Remove a chunk of values from the operation.
void	reprocessChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys chunkOk, Chunk<Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> keyChunk, IntChunk<Attributes.KeyIndices> bucketPositions, IntChunk<Attributes.ChunkPositions> runStartPositions, IntChunk<Attributes.ChunkLengths> runLengths)	Reprocess a chunk of data for a bucketed table.
void	reprocessChunk(UpdateByOperator.UpdateContext updateContext, OrderedKeys inputKeys, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> valuesChunk, ReadOnlyIndex postUpdateSourceIndex)	Reprocess a chunk of data for a non-bucketed table.
boolean	requiresKeys()	Query if the operator requires key values for the current stage.
boolean	requiresValues(UpdateByOperator.UpdateContext context)	Query if the operator requires values for the current stage.
void	resetForReprocess(UpdateByOperator.UpdateContext context, ReadOnlyIndex sourceIndex, long firstUnmodifiedKey)	Reset the operator to the state at the `firstModifiedKey` for non-bucketed operation.
void	resetForReprocess(UpdateByOperator.UpdateContext ctx, ReadOnlyIndex bucketIndex, long bucketPosition, long firstUnmodifiedKey)	Reset the operator to the state at the `firstModifiedKey` for the specified bucket.
void	setBucketCapacity(int capacity)	Notify the operator of the current maximum bucket.
void	setChunkSize(UpdateByOperator.UpdateContext context, int chunkSize)	Set the chunk size to be used for operations.
void	startTrackingPrev()	Indicate that the operation should start tracking previous values for ticking updates.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.illumon.iris.db.v2.updateby.UpdateByOperator

addChunk

Field Details

outputSource

protected final WritableSource<Byte> outputSource

maybeInnerSource

protected final WritableSource<Byte> maybeInnerSource

pair

protected final MatchPair pair

affectingColumns

protected final String[] affectingColumns

isRedirected

protected final boolean isRedirected

bucketLastVal

protected WritableSource<Byte> bucketLastVal

singletonVal

protected byte singletonVal

These are only used in grouped operations

singletonGroup

protected long singletonGroup

initialized

protected boolean initialized

Constructor Details

BaseByteUpdateByOperator

public BaseByteUpdateByOperator(@NotNull MatchPair pair, @NotNull String[] affectingColumns, @Nullable RedirectionIndex redirectionIndex)

Construct a base operator for operations that produce byte outputs.

Parameters:

pair - the MatchPair that defines the input/output for this operation

affectingColumns - a list of all columns (including the input column from the pair) that affects the result of this operator.

redirectionIndex - the RedirectionIndex if one is used

Method Details

getNullValue

protected byte getNullValue()

makeSparseSource

protected WritableSource<Byte> makeSparseSource()

makeDenseSource

protected WritableSource<Byte> makeDenseSource()

setChunkSize

public void setChunkSize(@NotNull UpdateByOperator.UpdateContext context, int chunkSize)

Description copied from interface: UpdateByOperator

Set the chunk size to be used for operations. This is used during the UpdateBy.UpdateType.Reprocess phase when the chunks allocated during the normal processing phase may not be large enough.

Specified by:

setChunkSize in interface UpdateByOperator

Parameters:

context - the context object

chunkSize - the new chunk size

setBucketCapacity

public void setBucketCapacity(int capacity)

Description copied from interface: UpdateByOperator

Notify the operator of the current maximum bucket.

Specified by:

setBucketCapacity in interface UpdateByOperator

Parameters:

capacity - the capacity

getInputColumnName

@NotNull public String getInputColumnName()

Description copied from interface: UpdateByOperator

Get the name of the input column this operator depends on.

Specified by:

getInputColumnName in interface UpdateByOperator

Returns:

the name of the input column

getAffectingColumnNames

@NotNull public String[] getAffectingColumnNames()

Description copied from interface: UpdateByOperator

Get an array of column names that, when modified, affect the result of this computation.

Specified by:

getAffectingColumnNames in interface UpdateByOperator

Returns:

an array of column names that affect this operator.

getOutputColumnNames

@NotNull public String[] getOutputColumnNames()

Description copied from interface: UpdateByOperator

Get an array of the output column names.

Specified by:

getOutputColumnNames in interface UpdateByOperator

Returns:

the output column names.

getOutputColumns

@NotNull public Map<String,ColumnSource<?>> getOutputColumns()

Description copied from interface: UpdateByOperator

Get a map of outputName to output ColumnSource for this operation.

Specified by:

getOutputColumns in interface UpdateByOperator

Returns:

a map of output column name to output column source

makeUpdateContext

@NotNull public UpdateByOperator.UpdateContext makeUpdateContext(int chunkSize)

Description copied from interface: UpdateByOperator

Make an UpdateByOperator.UpdateContext suitable for use with non-bucketed updates.

Specified by:

makeUpdateContext in interface UpdateByOperator

Parameters:

chunkSize - The expected size of chunks that will be provided during the update,

Returns:

a new context

initializeForUpdate

public void initializeForUpdate(@NotNull UpdateByOperator.UpdateContext context, @NotNull ShiftAwareListener.Update upstream, @NotNull ReadOnlyIndex resultSourceIndex, boolean usingBuckets, boolean isUpstreamAppendOnly)

Description copied from interface: UpdateByOperator

Initialize the operator for an update cycle. This is invoked before any other update processing occurs.

Specified by:

initializeForUpdate in interface UpdateByOperator

Parameters:

context - the context object

upstream - the upstream update to process

resultSourceIndex - the result index of the source table

usingBuckets - if the update is bucketed or not

isUpstreamAppendOnly - if the upstream update was detected to be append-only.

initializeFor

public void initializeFor(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull ReadOnlyIndex updateIndex, @NotNull UpdateBy.UpdateType type)

Description copied from interface: UpdateByOperator

Initialize the context for the specified stage of the update process. This will always be followed by a call to UpdateByOperator.finishFor(UpdateContext, UpdateType) at the end of each successful update.

Specified by:

initializeFor in interface UpdateByOperator

Parameters:

updateContext - the context object

updateIndex - the index of rows associated with the update.

type - the type of update being applied

finishFor

public void finishFor(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull UpdateBy.UpdateType type)

Description copied from interface: UpdateByOperator

Perform and bookkeeping required at the end of a single part of the update. This is always preceded with a call to UpdateByOperator.initializeFor(UpdateContext, ReadOnlyIndex, UpdateType)

Specified by:

finishFor in interface UpdateByOperator

Parameters:

updateContext - the context object

type - the type of update being applied

requiresKeys

public boolean requiresKeys()

Description copied from interface: UpdateByOperator

Query if the operator requires key values for the current stage. This method will always be invoked after an appropriate invocation of UpdateByOperator.initializeFor(UpdateContext, ReadOnlyIndex, UpdateType)

Specified by:

requiresKeys in interface UpdateByOperator

Returns:

true if the operator requires keys for this operation

requiresValues

public boolean requiresValues(@NotNull UpdateByOperator.UpdateContext context)

Description copied from interface: UpdateByOperator

Query if the operator requires values for the current stage.

Specified by:

requiresValues in interface UpdateByOperator

Parameters:

context - the context object

Returns:

true if values are required for compuitation

startTrackingPrev

public void startTrackingPrev()

Description copied from interface: UpdateByOperator

Indicate that the operation should start tracking previous values for ticking updates.

Specified by:

startTrackingPrev in interface UpdateByOperator

getAdditionalModifications

@NotNull public final ReadOnlyIndex getAdditionalModifications(@NotNull UpdateByOperator.UpdateContext ctx)

Description copied from interface: UpdateByOperator

Get an index of rows that were modified beyond the input set of modifications from the upstream. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed) if UpdateByOperator.anyModified(UpdateContext) has returned true.

Specified by:

getAdditionalModifications in interface UpdateByOperator

Parameters:

ctx - the context object

Returns:

a index of additional rows that were modified

anyModified

public final boolean anyModified(@NotNull UpdateByOperator.UpdateContext ctx)

Description copied from interface: UpdateByOperator

Check if the update has modified any rows for this operator. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed).

Specified by:

anyModified in interface UpdateByOperator

Parameters:

ctx - the context object

Returns:

true if the update modified any rows.

onBucketsRemoved

public void onBucketsRemoved(@NotNull ReadOnlyIndex removedBuckets)

Description copied from interface: UpdateByOperator

Called when some buckets have been completely emptied. Operators can use this to reset internal states.

Specified by:

onBucketsRemoved in interface UpdateByOperator

Parameters:

removedBuckets - an index of removed bucket positions.

canProcessNormalUpdate

public boolean canProcessNormalUpdate(@NotNull UpdateByOperator.UpdateContext context)

Description copied from interface: UpdateByOperator

Query if this operator can process the update normally, or if it can only reprocess. This method is guaranteed to be invoked after UpdateByOperator.initializeForUpdate(UpdateContext, ShiftAwareListener.Update, ReadOnlyIndex, boolean, boolean) so the operator is aware of the upstream ShiftAwareListener.Update.

Specified by:

canProcessNormalUpdate in interface UpdateByOperator

Parameters:

context - the context

Returns:

true if this operator can process the update normally.

addChunk

public void addChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull OrderedKeys inputKeys, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> values, long bucketPosition)

Description copied from interface: UpdateByOperator

Add a chunk of non-bucketed items to the operation.

Specified by:

addChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

keyChunk - a chunk of keys for the rows being added. If the operator returns false for UpdateByOperator.requiresKeys() this will be null.

values - the chunk of values for the rows being added

bucketPosition - the group position

doAddChunk

protected abstract void doAddChunk(@NotNull BaseByteUpdateByOperator.Context ctx, @NotNull OrderedKeys inputKeys, @NotNull Chunk<Attributes.Values> workingChunk, long bucketPosition)

Add a chunk of values to the operator.

Parameters:

ctx - the context object

inputKeys - the input keys for the chunk

workingChunk - the chunk of values

bucketPosition - the bucket position that the values belong to.

applyOutputShift

public void applyOutputShift(@NotNull UpdateByOperator.UpdateContext context, @NotNull ReadOnlyIndex subIndexToShift, long delta)

Description copied from interface: UpdateByOperator

Apply a shift to the operation.

Specified by:

applyOutputShift in interface UpdateByOperator

Parameters:

context - the context object

resetForReprocess

public void resetForReprocess(@NotNull UpdateByOperator.UpdateContext context, @NotNull ReadOnlyIndex sourceIndex, long firstUnmodifiedKey)

Description copied from interface: UpdateByOperator

Reset the operator to the state at the `firstModifiedKey` for non-bucketed operation. This is invoked immediately prior to calls to UpdateByOperator.reprocessChunk(UpdateContext, OrderedKeys, LongChunk, Chunk, ReadOnlyIndex).

A `firstUnmodifiedKey` of ReadOnlyIndex.NULL_KEY indicates that the entire table needs to be recomputed.

Specified by:

resetForReprocess in interface UpdateByOperator

Parameters:

context - the context object

sourceIndex - the current index of the source table

firstUnmodifiedKey - the first unmodified key after which we will reprocess rows.

resetForReprocess

public void resetForReprocess(@NotNull UpdateByOperator.UpdateContext ctx, @NotNull ReadOnlyIndex bucketIndex, long bucketPosition, long firstUnmodifiedKey)

Description copied from interface: UpdateByOperator

Reset the operator to the state at the `firstModifiedKey` for the specified bucket. This is invoked immediately prior to calls to UpdateByOperator.reprocessChunk(UpdateContext, OrderedKeys, Chunk, LongChunk, IntChunk, IntChunk, IntChunk).

Specified by:

resetForReprocess in interface UpdateByOperator

Parameters:

ctx - the context object

bucketIndex - the current index of the specified bucket

firstUnmodifiedKey - the first unmodified key in the bucket after which we will reprocess rows.

reprocessChunk

public void reprocessChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull OrderedKeys inputKeys, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> valuesChunk, @NotNull ReadOnlyIndex postUpdateSourceIndex)

Description copied from interface: UpdateByOperator

Reprocess a chunk of data for a non-bucketed table.

Specified by:

reprocessChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

inputKeys - the keys contained in the chunk

keyChunk - a LongChunk containing the keys if requested by UpdateByOperator.requiresKeys() or null.

valuesChunk - the current chunk of working values.

postUpdateSourceIndex - the resulting source index af

reprocessChunk

public void reprocessChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull OrderedKeys chunkOk, @NotNull Chunk<Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> keyChunk, @NotNull IntChunk<Attributes.KeyIndices> bucketPositions, @NotNull IntChunk<Attributes.ChunkPositions> runStartPositions, @NotNull IntChunk<Attributes.ChunkLengths> runLengths)

Description copied from interface: UpdateByOperator

Reprocess a chunk of data for a bucketed table.

Specified by:

reprocessChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

chunkOk - the keys contained in the chunk

values - the current chunk of working values.

keyChunk - a LongChunk containing the keys.

bucketPositions - a IntChunk containing the bucket position of each key. Parallel to `keyChunk` and `values

runStartPositions - the starting positions of each run within the key and value chunk

runLengths - the run runLengths of each run in the key and value chunks. Parallel to `runStartPositions`

modifyChunk

public final void modifyChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @Nullable LongChunk<Attributes.OrderedKeyIndices> prevKeyChunk, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> prevValuesChunk, @NotNull Chunk<Attributes.Values> postValuesChunk, long bucketPosition)

Description copied from interface: UpdateByOperator

Modify a chunk of values with the operation.

Specified by:

modifyChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

prevKeyChunk - a chunk of pre-shift keys. This will be equal to keyChunk if no shift is present

keyChunk - a chunk of post-shift space keys for the update.

prevValuesChunk - a chunk of previous values for the update

postValuesChunk - a chunk of current values for the update

bucketPosition - the position of the current group being processed

removeChunk

public final void removeChunk(@NotNull UpdateByOperator.UpdateContext updateContext, @Nullable LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> prevValuesChunk, long bucketPosition)

Description copied from interface: UpdateByOperator

Remove a chunk of values from the operation.

Specified by:

removeChunk in interface UpdateByOperator

Parameters:

updateContext - the context object

keyChunk - a chunk of keys being removed.

prevValuesChunk - the chunk of values being removed

bucketPosition - the position of the current group being processed

applyShift

public final void applyShift(@NotNull UpdateByOperator.UpdateContext updateContext, @NotNull ReadOnlyIndex prevIndex, @NotNull IndexShiftData shifted)

Description copied from interface: UpdateByOperator

Apply a shift to the operation.

Specified by:

applyShift in interface UpdateByOperator

Parameters:

updateContext - the context object

prevIndex - the pre-shifted index

shifted - the shifts being applied