Package com.illumon.iris.db.v2.sources

Class SparseArrayColumnSource<T>

java.lang.Object

com.illumon.iris.db.v2.sources.AbstractColumnSource<T>

com.illumon.iris.db.v2.sources.SparseArrayColumnSource<T>

All Implemented Interfaces:

ChunkSource<Attributes.Values>, ChunkSource.WithPrev<Attributes.Values>, DefaultChunkSource<Attributes.Values>, DefaultChunkSource.WithPrev<Attributes.Values>, FillContextMaker, GetContextMaker, ColumnSource<T>, ElementSource<T>, FillUnordered, Releasable, WritableChunkSink<Attributes.Values>, WritableSource<T>, TupleExporter<T>, TupleSource<T>, Serializable

Direct Known Subclasses:

BooleanSparseArraySource, ByteSparseArraySource, CharacterSparseArraySource, DoubleSparseArraySource, FloatSparseArraySource, IntegerSparseArraySource, LongSparseArraySource, NanosBasedTimeSparseArraySource, ObjectSparseArraySource, ShortSparseArraySource

public abstract class SparseArrayColumnSource<T>
extends AbstractColumnSource<T>
implements FillUnordered, WritableSource<T>

A column source backed by arrays that may not be filled in all blocks.

To store the blocks, we use a multi-level page table like structure. Each entry that exists is complete, i.e. we never reallocate partial blocks, we always allocate the complete block. The index key is divided as follows:

Description	Size	Bits
Block 0	19	62-44
Block 1	18	43-26
Block 2	18	25-8
Index Within Block	8	7-0

Bit 63, the sign bit, is used to indicate null (that is, all negative numbers are defined to be null)

Parallel structures are used for previous values and prevInUse. We recycle all levels of the previous blocks, so that the previous structure takes up memory only while it is in use.

See Also:

• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected static class	SparseArrayColumnSource.FillByContext<UArray>

Nested classes/interfaces inherited from class com.illumon.iris.db.v2.sources.AbstractColumnSource

AbstractColumnSource.DefaultedImmutable<DATA_TYPE>, AbstractColumnSource.DefaultedMutable<DATA_TYPE>, AbstractColumnSource.IsSerializable

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.sources.chunk.ChunkSource

ChunkSource.FillContext, ChunkSource.GetContext, ChunkSource.WithPrev<ATTR extends Attributes.Any>

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.sources.chunk.DefaultChunkSource

DefaultChunkSource.SupportsContiguousGet<ATTR extends Attributes.Any>, DefaultChunkSource.WithPrev<ATTR extends Attributes.Any>

Nested classes/interfaces inherited from interface com.illumon.iris.db.v2.sources.WritableChunkSink

WritableChunkSink.FillFromContext

Field Summary

Fields

Modifier and Type	Field	Description
static final SparseArrayColumnSource[]	ZERO_LENGTH_SPARSE_ARRAY_COLUMN_SOURCE_ARRAY

Fields inherited from class com.illumon.iris.db.v2.sources.AbstractColumnSource

componentType, type

Fields inherited from interface com.illumon.iris.db.v2.sources.chunk.ChunkSource

DEFAULT_FILL_INSTANCE, ZERO_LENGTH_CHUNK_SOURCE_ARRAY

Fields inherited from interface com.illumon.iris.db.v2.sources.chunk.ChunkSource.WithPrev

ZERO_LENGTH_CHUNK_SOURCE_WITH_PREV_ARRAY

Fields inherited from interface com.illumon.iris.db.v2.sources.ColumnSource

ZERO_LENGTH_COLUMN_SOURCE_ARRAY

Fields inherited from interface com.illumon.iris.db.v2.sources.WritableChunkSink

DEFAULT_FILL_FROM_INSTANCE

Fields inherited from interface com.illumon.iris.db.v2.sources.WritableSource

ZERO_LENGTH_WRITABLE_SOURCE_ARRAY

Method Summary

Modifier and Type	Method	Description
void	fillChunk(ChunkSource.FillContext context, WritableChunk<? super Attributes.Values> dest, OrderedKeys orderedKeys)	Populates the given destination chunk with data corresponding to the keys from the given OrderedKeys.
void	fillChunkUnordered(ChunkSource.FillContext context, WritableChunk<? super Attributes.Values> dest, LongChunk<? extends Attributes.KeyIndices> keys)	Populates a contiguous portion of the given destination chunk with data corresponding to the keys from the given LongChunk.
void	fillFromChunk(WritableChunkSink.FillFromContext context, Chunk<? extends Attributes.Values> src, OrderedKeys orderedKeys)	Our default, inefficient, implementation.
void	fillPrevChunkUnordered(ChunkSource.FillContext context, WritableChunk<? super Attributes.Values> dest, LongChunk<? extends Attributes.KeyIndices> keys)	Populates a contiguous portion of the given destination chunk with prev data corresponding to the keys from the given LongChunk.
static SparseArrayColumnSource<DBDateTime>	getDateTimeMemoryColumnSource(long[] data)
int	getPreferredChunkSize()	Using a preferred chunk size of BLOCK_SIZE gives us the opportunity to directly return chunks from our data structure rather than copying data.
static SparseArrayColumnSource<Byte>	getSparseMemoryColumnSource(byte[] data)
static SparseArrayColumnSource<Character>	getSparseMemoryColumnSource(char[] data)
static SparseArrayColumnSource<Double>	getSparseMemoryColumnSource(double[] data)
static SparseArrayColumnSource<Float>	getSparseMemoryColumnSource(float[] data)
static SparseArrayColumnSource<Integer>	getSparseMemoryColumnSource(int[] data)
static SparseArrayColumnSource<Long>	getSparseMemoryColumnSource(long[] data)
static <T> SparseArrayColumnSource<T>	getSparseMemoryColumnSource(long size, Class<T> type)
static <T> SparseArrayColumnSource<T>	getSparseMemoryColumnSource(long size, Class<T> type, Class componentType)
static SparseArrayColumnSource<Short>	getSparseMemoryColumnSource(short[] data)
static <T> SparseArrayColumnSource<T>	getSparseMemoryColumnSource(Class<T> type)
static <T> SparseArrayColumnSource<T>	getSparseMemoryColumnSource(Class<T> type, Class componentType)
static ColumnSource	getSparseMemoryColumnSource(Object dataArray)
static <T> SparseArrayColumnSource<T>	getSparseMemoryColumnSource(Collection<T> data, Class<T> type)
boolean	isImmutable()	Determine if this column source is immutable, meaning that the values at a given index key never change.
WritableChunkSink.FillFromContext	makeFillFromContext(int chunkCapacity)	Provide a default, empty WritableChunkSink.FillFromContext for use with our default WritableSource.fillFromChunk(com.illumon.iris.db.v2.sources.WritableChunkSink.FillFromContext, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.utils.OrderedKeys).
void	remove(Index toRemove)
void	set(long key, byte value)
void	set(long key, char value)
void	set(long key, double value)
void	set(long key, float value)
void	set(long key, int value)
void	set(long key, long value)
void	set(long key, short value)
void	setImmutable()
void	setNull(OrderedKeys orderedKeys)
void	shift(ReadOnlyIndex keysToShift, long shiftDelta)

Methods inherited from class com.illumon.iris.db.v2.sources.AbstractColumnSource

allowsReinterpret, canUseGrouping, defaultFillChunk, doReinterpret, fillPrevChunk, getComponentType, getGroupingBuilder, getGroupingProvider, getNativeType, getType, hasGrouping, isSerializable, match, matchWithGrouping, reinterpret, setGroupingProvider

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface com.illumon.iris.db.v2.sources.chunk.ChunkSource

getChunkType

Methods inherited from interface com.illumon.iris.db.v2.sources.chunk.ChunkSource.WithPrev

fillPrevChunk

Methods inherited from interface com.illumon.iris.db.v2.sources.ColumnSource

allowsReinterpret, cast, createPreviousTuple, createTuple, createTupleFromValues, exportElement, exportElement, exportToExternalKey, getColumnSources, getComponentType, getGroupingBuilder, getGroupingProvider, getNativeType, getPrevSource, getType, hasGrouping, match, reinterpret, releaseCachedResources, setGroupingProvider, startTrackingPrevValues

Methods inherited from interface com.illumon.iris.db.v2.sources.chunk.DefaultChunkSource

getChunk, getChunk, getChunkByFilling, makeFillContext, makeGetContext

Methods inherited from interface com.illumon.iris.db.v2.sources.chunk.DefaultChunkSource.WithPrev

getPrevChunk, getPrevChunk, getPrevChunkByFilling

Methods inherited from interface com.illumon.iris.db.v2.sources.ElementSource

get, getBoolean, getByte, getChar, getDouble, getFloat, getInt, getLong, getPrev, getPrevBoolean, getPrevByte, getPrevChar, getPrevDouble, getPrevFloat, getPrevInt, getPrevLong, getPrevShort, getShort

Methods inherited from interface com.illumon.iris.db.v2.sources.chunk.FillContextMaker

makeFillContext

Methods inherited from interface com.illumon.iris.db.v2.sources.chunk.GetContextMaker

makeGetContext

Methods inherited from interface com.illumon.iris.db.v2.tuples.TupleExporter

exportElementReinterpreted

Methods inherited from interface com.illumon.iris.db.v2.tuples.TupleSource

createTupleFromReinterpretedValues

Methods inherited from interface com.illumon.iris.db.v2.sources.WritableSource

copy, ensureCapacity, fillFromChunkUnordered, set, setNull

Field Details

ZERO_LENGTH_SPARSE_ARRAY_COLUMN_SOURCE_ARRAY

public static final SparseArrayColumnSource[] ZERO_LENGTH_SPARSE_ARRAY_COLUMN_SOURCE_ARRAY

Method Details

set

public void set(long key,
 byte value)

Specified by:

set in interface WritableSource<T>

set

public void set(long key,
 char value)

Specified by:

set in interface WritableSource<T>

set

public void set(long key,
 double value)

Specified by:

set in interface WritableSource<T>

set

public void set(long key,
 float value)

Specified by:

set in interface WritableSource<T>

set

public void set(long key,
 int value)

Specified by:

set in interface WritableSource<T>

set

public void set(long key,
 long value)

Specified by:

set in interface WritableSource<T>

set

public void set(long key,
 short value)

Specified by:

set in interface WritableSource<T>

shift

public void shift(ReadOnlyIndex keysToShift,
 long shiftDelta)

remove

public void remove(Index toRemove)

getSparseMemoryColumnSource

public static <T> SparseArrayColumnSource<T> getSparseMemoryColumnSource(Collection<T> data,
 Class<T> type)

getSparseMemoryColumnSource

public static SparseArrayColumnSource<Byte> getSparseMemoryColumnSource(byte[] data)

getSparseMemoryColumnSource

public static SparseArrayColumnSource<Character> getSparseMemoryColumnSource(char[] data)

getSparseMemoryColumnSource

public static SparseArrayColumnSource<Double> getSparseMemoryColumnSource(double[] data)

getSparseMemoryColumnSource

public static SparseArrayColumnSource<Float> getSparseMemoryColumnSource(float[] data)

getSparseMemoryColumnSource

public static SparseArrayColumnSource<Integer> getSparseMemoryColumnSource(int[] data)

getSparseMemoryColumnSource

public static SparseArrayColumnSource<Long> getSparseMemoryColumnSource(long[] data)

getDateTimeMemoryColumnSource

public static SparseArrayColumnSource<DBDateTime> getDateTimeMemoryColumnSource(long[] data)

getSparseMemoryColumnSource

public static SparseArrayColumnSource<Short> getSparseMemoryColumnSource(short[] data)

getSparseMemoryColumnSource

public static <T> SparseArrayColumnSource<T> getSparseMemoryColumnSource(Class<T> type)

getSparseMemoryColumnSource

public static <T> SparseArrayColumnSource<T> getSparseMemoryColumnSource(Class<T> type,
 Class componentType)

getSparseMemoryColumnSource

public static <T> SparseArrayColumnSource<T> getSparseMemoryColumnSource(long size,
 Class<T> type)

getSparseMemoryColumnSource

public static <T> SparseArrayColumnSource<T> getSparseMemoryColumnSource(long size,
 Class<T> type,
 @Nullable
 Class componentType)

getSparseMemoryColumnSource

public static ColumnSource getSparseMemoryColumnSource(Object dataArray)

getPreferredChunkSize

public int getPreferredChunkSize()

Using a preferred chunk size of BLOCK_SIZE gives us the opportunity to directly return chunks from our data structure rather than copying data.

fillChunk

public void fillChunk(@NotNull
 ChunkSource.FillContext context,
 @NotNull
 WritableChunk<? super Attributes.Values> dest,
 @NotNull
 OrderedKeys orderedKeys)

Description copied from interface: ChunkSource

Populates the given destination chunk with data corresponding to the keys from the given OrderedKeys.

Specified by:

fillChunk in interface ChunkSource<T>

Overrides:

fillChunk in class AbstractColumnSource<T>

Parameters:

context - A context containing all mutable/state related data used in retrieving the Chunk.

dest - The chunk to be populated according to orderedKeys. No assumptions shall be made about the size of the chunk shall be made. The chunk will be populated from position [0,orderedKeys.size()).

orderedKeys - An OrderedKeys representing the keys to be fetched

setNull

public void setNull(OrderedKeys orderedKeys)

Specified by:

setNull in interface WritableSource<T>

fillChunkUnordered

public void fillChunkUnordered(@NotNull
 ChunkSource.FillContext context,
 @NotNull
 WritableChunk<? super Attributes.Values> dest,
 @NotNull
 LongChunk<? extends Attributes.KeyIndices> keys)

Description copied from interface: FillUnordered

Populates a contiguous portion of the given destination chunk with data corresponding to the keys from the given LongChunk.

Specified by:

fillChunkUnordered in interface FillUnordered

Parameters:

context - A context containing all mutable/state related data used in retrieving the Chunk.

dest - The chunk to be populated according to keys

keys - A chunk of individual, not assumed to be ordered keys to be fetched

fillPrevChunkUnordered

public void fillPrevChunkUnordered(@NotNull
 ChunkSource.FillContext context,
 @NotNull
 WritableChunk<? super Attributes.Values> dest,
 @NotNull
 LongChunk<? extends Attributes.KeyIndices> keys)

Description copied from interface: FillUnordered

Populates a contiguous portion of the given destination chunk with prev data corresponding to the keys from the given LongChunk.

Specified by:

fillPrevChunkUnordered in interface FillUnordered

Parameters:

context - A context containing all mutable/state related data used in retrieving the Chunk.

dest - The chunk to be populated according to keys

keys - A chunk of individual, not assumed to be ordered keys to be fetched

makeFillFromContext

public WritableChunkSink.FillFromContext makeFillFromContext(int chunkCapacity)

Description copied from interface: WritableSource

Provide a default, empty WritableChunkSink.FillFromContext for use with our default WritableSource.fillFromChunk(com.illumon.iris.db.v2.sources.WritableChunkSink.FillFromContext, com.illumon.iris.db.v2.sources.chunk.Chunk<? extends com.illumon.iris.db.v2.sources.chunk.Attributes.Values>, com.illumon.iris.db.v2.utils.OrderedKeys).

Specified by:

makeFillFromContext in interface WritableChunkSink<T>

Specified by:

makeFillFromContext in interface WritableSource<T>

fillFromChunk

public void fillFromChunk(@NotNull
 WritableChunkSink.FillFromContext context,
 @NotNull
 Chunk<? extends Attributes.Values> src,
 @NotNull
 OrderedKeys orderedKeys)

Description copied from interface: WritableSource

Our default, inefficient, implementation. Inheritors who care should provide a better implementation.

Specified by:

fillFromChunk in interface WritableChunkSink<T>

Specified by:

fillFromChunk in interface WritableSource<T>

Parameters:

context - A context containing all mutable/state related data used in writing the Chunk.

src - The source of the data orderedKeys

orderedKeys - An OrderedKeys representing the keys to be written

isImmutable

public boolean isImmutable()

Description copied from interface: ColumnSource

Determine if this column source is immutable, meaning that the values at a given index key never change.

Specified by:

isImmutable in interface ColumnSource<T>

Returns:

true if the values at a given index of the column source never change, false otherwise

setImmutable

public void setImmutable()