Package com.illumon.iris.db.v2.updateby.fill

Class BaseDynamicFillByOperator

java.lang.Object

com.illumon.iris.db.v2.updateby.fill.BaseDynamicFillByOperator

All Implemented Interfaces:

ChunkedUpdateByOperator

Direct Known Subclasses:

BooleanChunkedFillByOperator, ByteChunkedFillByOperator, CharChunkedFillByOperator, DoubleChunkedFillByOperator, FloatChunkedFillByOperator, IntChunkedFillByOperator, LongChunkedFillByOperator, ObjectChunkedFillByOperator, ShortChunkedFillByOperator

public abstract class BaseDynamicFillByOperator
extends Object
implements ChunkedUpdateByOperator

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	BaseDynamicFillByOperator.SingletonContext

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.updateby.ChunkedUpdateByOperator

ChunkedUpdateByOperator.BucketedUpdateContext, ChunkedUpdateByOperator.UpdateContext

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	initialized
protected RedirectionIndex	redirection

Fields inherited from interface com.illumon.iris.db.v2.updateby.ChunkedUpdateByOperator

ZERO_LENGTH_OP_ARRAY

Method Summary

Modifier and Type	Method	Description
void	addChunk(ChunkedUpdateByOperator.BucketedUpdateContext context, Chunk<Attributes.Values> values, LongChunk<? extends Attributes.KeyIndices> keyChunk, IntChunk<Attributes.KeyIndices> bucketPositions, IntChunk<Attributes.ChunkPositions> startPositions, IntChunk<Attributes.ChunkLengths> length)	Add a chunk of bucketed items to the operation.
boolean	anyModified(ChunkedUpdateByOperator.BucketedUpdateContext inCtx)	Check if the update has modified any rows for this operator.
boolean	anyModified(ChunkedUpdateByOperator.UpdateContext inCtx)	Check if the update has modified any rows for this operator.
void	applyShift(ChunkedUpdateByOperator.UpdateContext inCtx, ReadOnlyIndex toShift, IndexShiftData shiftData)	Apply the specified shifts to this operator.
void	finishFor(ChunkedUpdateByOperator.BucketedUpdateContext inCtx, UpdateBy.UpdateType type)	Perform and bookkeeping required at the end of a single part of the update.
void	finishFor(ChunkedUpdateByOperator.UpdateContext inCtx, UpdateBy.UpdateType type)	Perform and bookkeeping required at the end of a single part of the update.
ReadOnlyIndex	getAdditionalModifications(ChunkedUpdateByOperator.UpdateContext inCtx)	Get an index of rows that were modified beyond the input set of modifications from the upstream.
String	getInputColumnName()	Get the name of the input column this operator depends on.
String[]	getOutputColumnNames()	Get an array of the output column names.
Map<String,ColumnSource<?>>	getOutputColumns()	Get a map of outputName to output ColumnSource for this operation.
void	initializeFor(ChunkedUpdateByOperator.BucketedUpdateContext inCtx, UpdateBy.UpdateType type, ReadOnlyIndex updateIndex)	Initialize the context for the specified stage of the update process.
void	initializeFor(ChunkedUpdateByOperator.UpdateContext inCtx, UpdateBy.UpdateType type, ReadOnlyIndex updateIndex)	Initialize the context for the specified stage of the update process.
ChunkedUpdateByOperator.BucketedUpdateContext	makeBucketedContext(int chunkSize)	Make an ChunkedUpdateByOperator.UpdateContext suitable for use with bucketed updates.
ChunkedUpdateByOperator.UpdateContext	makeUpdateContext(int chunkSize)	Make an ChunkedUpdateByOperator.UpdateContext suitable for use with non-bucketed updates.
void	removeChunk(ChunkedUpdateByOperator.UpdateContext inCtx, LongChunk<Attributes.OrderedKeyIndices> keyChunk, Chunk<Attributes.Values> previousValuesChunk, long groupPosition)	Remove a chunk of values.
void	setBucketCapacity(int intValue)	Notify the operator of the current maximum bucket.
void	setUpstream(ChunkedUpdateByOperator.BucketedUpdateContext inCtx, ShiftAwareListener.Update upstream)	Set the upstream update object for this update cycle.
void	setUpstream(ChunkedUpdateByOperator.UpdateContext inCtx, ShiftAwareListener.Update upstream)	Set the upstream update object for this update cycle.
void	startTrackingPrev()	Indicate that the operation should start tracking previous values for ticking updates.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.illumon.iris.db.v2.updateby.ChunkedUpdateByOperator

addChunk, modifyChunk

Field Details

redirection

protected final RedirectionIndex redirection

initialized

protected boolean initialized

Method Details

setBucketCapacity

public void setBucketCapacity(int intValue)

Description copied from interface: ChunkedUpdateByOperator

Notify the operator of the current maximum bucket.

Specified by:

setBucketCapacity in interface ChunkedUpdateByOperator

Parameters:

intValue - the capacity

startTrackingPrev

public void startTrackingPrev()

Description copied from interface: ChunkedUpdateByOperator

Indicate that the operation should start tracking previous values for ticking updates.

Specified by:

startTrackingPrev in interface ChunkedUpdateByOperator

getInputColumnName

@NotNull public String getInputColumnName()

Description copied from interface: ChunkedUpdateByOperator

Get the name of the input column this operator depends on.

Specified by:

getInputColumnName in interface ChunkedUpdateByOperator

Returns:

the name of the input column

getOutputColumnNames

@NotNull public String[] getOutputColumnNames()

Description copied from interface: ChunkedUpdateByOperator

Get an array of the output column names.

Specified by:

getOutputColumnNames in interface ChunkedUpdateByOperator

Returns:

the output column names.

getOutputColumns

@NotNull public Map<String,ColumnSource<?>> getOutputColumns()

Description copied from interface: ChunkedUpdateByOperator

Get a map of outputName to output ColumnSource for this operation.

Specified by:

getOutputColumns in interface ChunkedUpdateByOperator

Returns:

a map of output column name to output column source

makeUpdateContext

@NotNull public ChunkedUpdateByOperator.UpdateContext makeUpdateContext(int chunkSize)

Description copied from interface: ChunkedUpdateByOperator

Make an ChunkedUpdateByOperator.UpdateContext suitable for use with non-bucketed updates.

Specified by:

makeUpdateContext in interface ChunkedUpdateByOperator

Parameters:

chunkSize - The expected size of chunks that will be provided during the update,

Returns:

a new updateContext

makeBucketedContext

@NotNull public ChunkedUpdateByOperator.BucketedUpdateContext makeBucketedContext(int chunkSize)

Description copied from interface: ChunkedUpdateByOperator

Make an ChunkedUpdateByOperator.UpdateContext suitable for use with bucketed updates.

Specified by:

makeBucketedContext in interface ChunkedUpdateByOperator

Parameters:

chunkSize - The expected size of chunks that will be provided during the update,

Returns:

a new updateContext

initializeFor

public void initializeFor(@NotNull ChunkedUpdateByOperator.UpdateContext inCtx, @NotNull UpdateBy.UpdateType type, @NotNull ReadOnlyIndex updateIndex)

Description copied from interface: ChunkedUpdateByOperator

Initialize the context for the specified stage of the update process. This will always be followed by a call to ChunkedUpdateByOperator.finishFor(UpdateContext, UpdateBy.UpdateType) at the end of each successful update.

Specified by:

initializeFor in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

type - the type of update being applied

updateIndex - the index of rows associated with the update.

initializeFor

public void initializeFor(@NotNull ChunkedUpdateByOperator.BucketedUpdateContext inCtx, @NotNull UpdateBy.UpdateType type, @NotNull ReadOnlyIndex updateIndex)

Description copied from interface: ChunkedUpdateByOperator

Initialize the context for the specified stage of the update process. This will always be followed by a call to ChunkedUpdateByOperator.finishFor(UpdateContext, UpdateBy.UpdateType) at the end of each successful update.

Specified by:

initializeFor in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

type - the type of update being applied

updateIndex - the index of rows associated with the update.

finishFor

public void finishFor(@NotNull ChunkedUpdateByOperator.UpdateContext inCtx, @NotNull UpdateBy.UpdateType type)

Description copied from interface: ChunkedUpdateByOperator

Perform and bookkeeping required at the end of a single part of the update. This is always preceded with a call to ChunkedUpdateByOperator.initializeFor(UpdateContext, UpdateBy.UpdateType, ReadOnlyIndex)

Specified by:

finishFor in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

type - the type of update being applied

finishFor

public void finishFor(@NotNull ChunkedUpdateByOperator.BucketedUpdateContext inCtx, @NotNull UpdateBy.UpdateType type)

Description copied from interface: ChunkedUpdateByOperator

Perform and bookkeeping required at the end of a single part of the update. This is always preceded with a call to ChunkedUpdateByOperator.initializeFor(UpdateContext, UpdateBy.UpdateType, ReadOnlyIndex)

Specified by:

finishFor in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

type - the type of update being applied

anyModified

public boolean anyModified(@NotNull ChunkedUpdateByOperator.UpdateContext inCtx)

Description copied from interface: ChunkedUpdateByOperator

Check if the update has modified any rows for this operator. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed).

Specified by:

anyModified in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

Returns:

true if the update modified any rows.

anyModified

public boolean anyModified(@NotNull ChunkedUpdateByOperator.BucketedUpdateContext inCtx)

Description copied from interface: ChunkedUpdateByOperator

Check if the update has modified any rows for this operator. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed).

Specified by:

anyModified in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

Returns:

true if the update modified any rows.

getAdditionalModifications

@NotNull public ReadOnlyIndex getAdditionalModifications(@NotNull ChunkedUpdateByOperator.UpdateContext inCtx)

Description copied from interface: ChunkedUpdateByOperator

Get an index of rows that were modified beyond the input set of modifications from the upstream. This is invoked once at the end of a complete update cycle (that is, after all adds, removes, modifies and shifts have been processed) if ChunkedUpdateByOperator.anyModified(UpdateContext) has returned true.

Specified by:

getAdditionalModifications in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

Returns:

a index of additional rows that were modified

setUpstream

public void setUpstream(@NotNull ChunkedUpdateByOperator.UpdateContext inCtx, @NotNull ShiftAwareListener.Update upstream)

Description copied from interface: ChunkedUpdateByOperator

Set the upstream update object for this update cycle.

Specified by:

setUpstream in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

upstream - the upstream update

setUpstream

public void setUpstream(@NotNull ChunkedUpdateByOperator.BucketedUpdateContext inCtx, @NotNull ShiftAwareListener.Update upstream)

Description copied from interface: ChunkedUpdateByOperator

Set the upstream update object for this update cycle.

Specified by:

setUpstream in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

upstream - the upstream update

applyShift

public void applyShift(@NotNull ChunkedUpdateByOperator.UpdateContext inCtx, @NotNull ReadOnlyIndex toShift, @NotNull IndexShiftData shiftData)

Apply the specified shifts to this operator. This has a few steps

1: In order to shift the redirection properly, we have to visit each shifted range and inspect the keys that are affected. If the value of a particular redirection fell within the shifted range, it too must be adjusted. If it did not, then we leave it alone.

It is worth noting that while we can create a single forward search iterator to visit affected keys that are being shifted to the left (negative delta), we can't do the same for right shifts (positive delta). We need to iterate the affected keys in reverse order, but advance in normal order, so we have to create a new reverse iterator on each separate right shifted range

2: After the redirection has been shifted, we can then apply the shift to the redirected row index. Then we have to shift any accumulated modified rows in the context

3: Finally, we have to repair any redirections that were broken by shifting. This is easy because we know that broken redirections will end up in between shifted ranges. So we can start at the first redirection after the first shifted range. Since shifts don't change data, we can locate the last valid row before the next redirection index using a ReadOnlyIndex.SearchIterator. We will fix each indirection that it incorrect until we find a good one, then move the ShiftData position to the first range past the last redirection, and repeat the process until we are done.

Specified by:

applyShift in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

toShift - the set of rows that can be shifted

shiftData - the shift data

addChunk

public void addChunk(@NotNull ChunkedUpdateByOperator.BucketedUpdateContext context, @NotNull Chunk<Attributes.Values> values, @NotNull LongChunk<? extends Attributes.KeyIndices> keyChunk, @NotNull IntChunk<Attributes.KeyIndices> bucketPositions, @NotNull IntChunk<Attributes.ChunkPositions> startPositions, @NotNull IntChunk<Attributes.ChunkLengths> length)

Description copied from interface: ChunkedUpdateByOperator

Add a chunk of bucketed items to the operation.

Specified by:

addChunk in interface ChunkedUpdateByOperator

Parameters:

context - the context object

values - the value chunk

keyChunk - a chunk of keys for the rows being added

bucketPositions - a chunk of hash bucket positions for each key

startPositions - the start position of a run within the chunk

length - the length of each run of bucket values

removeChunk

public void removeChunk(@NotNull ChunkedUpdateByOperator.UpdateContext inCtx, @NotNull LongChunk<Attributes.OrderedKeyIndices> keyChunk, @NotNull Chunk<Attributes.Values> previousValuesChunk, long groupPosition)

Remove a chunk of values. This is very simple, if the key being removed affect redirections to the right, simply find the previous valid row before the one being removed and propagate it forward to all the subsequent rows that were directed to this one.

Specified by:

removeChunk in interface ChunkedUpdateByOperator

Parameters:

inCtx - the context object

keyChunk - the chunk of keys being removed.

groupPosition -

previousValuesChunk - the chunk of values being removed