Package io.deephaven.extensions.barrage.table

Class BarrageTable

java.lang.Object

io.deephaven.util.referencecounting.ReferenceCounted

io.deephaven.engine.liveness.ReferenceCountedLivenessReferent

io.deephaven.engine.liveness.ReferenceCountedLivenessNode

io.deephaven.engine.liveness.LivenessArtifact

io.deephaven.engine.table.impl.LiveAttributeMap<IFACE_TYPE,IMPL_TYPE>

io.deephaven.engine.table.impl.BaseGridAttributes<Table,IMPL_TYPE>

io.deephaven.engine.table.impl.BaseTable<QueryTable>

io.deephaven.engine.table.impl.QueryTable

io.deephaven.extensions.barrage.table.BarrageTable

All Implemented Interfaces:

TableOperations<Table,Table>, TableOperationsDefaults<Table,Table>, LogOutputAppendable, LivenessManager, LivenessNode, LivenessReferent, AttributeMap<Table>, GridAttributes<Table>, HasParentPerformanceIds, HasRefreshingSource, NotificationStepReceiver, NotificationStepSource, TableDefaults, BarrageMessage.Listener, Table, DynamicNode, NotificationQueue.Dependency, SystemicObject<Table>, LongSizedDataStructure, Serializable

Direct Known Subclasses:

BarrageBlinkTable, BarrageRedirectedTable

public abstract class BarrageTable
extends QueryTable
implements BarrageMessage.Listener, HasRefreshingSource

A client side Table that mirrors an upstream/server side Table.

Note that viewports are defined in row positions of the upstream table.

See Also:

• Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	BarrageTable.Stats
static interface	BarrageTable.ViewportChangedCallback

Nested classes/interfaces inherited from class io.deephaven.engine.table.impl.QueryTable

QueryTable.FilteredTable, QueryTable.MemoizableOperation<T extends DynamicNode & NotificationStepReceiver>, QueryTable.Operation<T extends DynamicNode & NotificationStepReceiver>

Nested classes/interfaces inherited from class io.deephaven.engine.table.impl.BaseTable

BaseTable.CopyAttributeOperation, BaseTable.ListenerImpl, BaseTable.ShiftObliviousListenerImpl, BaseTable.SnapshotControlFactory<T extends ConstructSnapshot.SnapshotControl>

Field Summary

Fields

Modifier and Type	Field	Description
protected static final int	BATCH_SIZE
protected long	capacity	the capacity that the destSources been set to
static final boolean	DEBUG_ENABLED
protected final WritableColumnSource<?>[]	destSources	the reinterpreted destination writable sources
protected static final Logger	log
protected final BarrageTable.Stats	stats

Fields inherited from class io.deephaven.engine.table.impl.QueryTable

DATA_INDEX_FOR_WHERE_THRESHOLD, DISABLE_WHERE_PUSHDOWN_DATA_INDEX, DISABLE_WHERE_PUSHDOWN_PARQUET_DICTIONARY, DISABLE_WHERE_PUSHDOWN_PARQUET_ROW_GROUP_METADATA, ENABLE_PARALLEL_SNAPSHOT, MINIMUM_PARALLEL_SELECT_ROWS, MINIMUM_PARALLEL_SNAPSHOT_ROWS, SERIAL_SELECT_IMPLICIT_BARRIERS, STATELESS_FILTERS_BY_DEFAULT, STATELESS_SELECT_BY_DEFAULT, USE_DATA_INDEX_FOR_AGGREGATION, USE_DATA_INDEX_FOR_JOINS, USE_DATA_INDEX_FOR_WHERE

Fields inherited from class io.deephaven.engine.table.impl.BaseTable

definition, PRINT_SERIALIZED_UPDATE_OVERLAPS, updateGraph, VALIDATE_UPDATE_OVERLAPS

Fields inherited from interface io.deephaven.engine.table.GridAttributes

COLUMN_DESCRIPTIONS_ATTRIBUTE, DESCRIPTION_ATTRIBUTE, LAYOUT_HINTS_ATTRIBUTE, SORTABLE_COLUMNS_ATTRIBUTE

Fields inherited from interface io.deephaven.engine.table.impl.NotificationStepReceiver

NULL_NOTIFICATION_STEP

Fields inherited from interface io.deephaven.engine.table.Table

ADD_ONLY_TABLE_ATTRIBUTE, AGGREGATION_ROW_LOOKUP_ATTRIBUTE, APPEND_ONLY_TABLE_ATTRIBUTE, BARRAGE_PERFORMANCE_KEY_ATTRIBUTE, BARRAGE_SCHEMA_ATTRIBUTE, BLINK_TABLE_ATTRIBUTE, FILTERABLE_COLUMNS_ATTRIBUTE, INITIALLY_EMPTY_COALESCED_SOURCE_TABLE_ATTRIBUTE, INPUT_TABLE_ATTRIBUTE, KEY_COLUMNS_ATTRIBUTE, MERGED_TABLE_ATTRIBUTE, NON_DISPLAY_TABLE, PLUGIN_NAME, PREVIEW_PARENT_TABLE, SNAPSHOT_VIEWPORT_TYPE, SORT_REVERSE_LOOKUP_ATTRIBUTE, SORT_ROW_REDIRECTION_ATTRIBUTE, SORTED_COLUMNS_ATTRIBUTE, SYSTEMIC_TABLE_ATTRIBUTE, TEST_SOURCE_TABLE_ATTRIBUTE, TOTALS_TABLE_ATTRIBUTE, UNIQUE_KEYS_ATTRIBUTE

Fields inherited from interface io.deephaven.engine.table.impl.TableDefaults

ZERO_LENGTH_TABLE_ARRAY

Fields inherited from interface io.deephaven.api.TableOperations

AGG_BY_PRESERVE_EMPTY_DEFAULT

Fields inherited from interface io.deephaven.api.TableOperationsDefaults

ZERO_LENGTH_COLUMNNAME_ARRAY, ZERO_LENGTH_FILTER_ARRAY

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BarrageTable(@Nullable String channelName, UpdateSourceRegistrar registrar, NotificationQueue notificationQueue, @Nullable ScheduledExecutorService executorService, LinkedHashMap<String,ColumnSource<?>> columns, WritableColumnSource<?>[] writableSources, Map<String,Object> attributes, @Nullable BarrageTable.ViewportChangedCallback viewportChangedCallback)

Method Summary

Modifier and Type	Method	Description
void	addSourceToRegistrar()	Add this table to the registrar so that it can be refreshed.
protected abstract TableUpdate	applyUpdates(ArrayDeque<BarrageMessage> localPendingUpdates)
protected LogEntry	beginLog(LogLevel level)	Convenience method for writing consistent log messages from this object.
protected void	destroy()	Attempt to release (destructively when necessary) resources held by this object.
protected void	doWakeup()
@Nullable Object	getAttribute(@NotNull String key)	Get the value for the specified attribute key.
String	getDescription()
LongConsumer	getDeserializationTmConsumer()
protected NotificationQueue	getNotificationQueue()	Get the notification queue to insert notifications into as they are generated by listeners during BaseTable.notifyListeners(io.deephaven.engine.rowset.RowSet, io.deephaven.engine.rowset.RowSet, io.deephaven.engine.rowset.RowSet) and BaseTable.notifyListenersOnError(Throwable, TableListener.Entry).
BitSet	getServerColumns()
boolean	getServerReverseViewport()
RowSet	getServerViewport()
ChunkType[]	getWireChunkTypes()
Class<?>[]	getWireComponentTypes()
Class<?>[]	getWireTypes()
void	handleBarrageError(Throwable t)
void	handleBarrageMessage(BarrageMessage update)
protected boolean	isSubscribedColumn(int i)
static BarrageTable	make(@Nullable String channelName, @Nullable ScheduledExecutorService executorService, BarrageUtil.ConvertedArrowSchema schema, boolean isFullSubscription, @Nullable BarrageTable.ViewportChangedCallback vpCallback)	Set up a replicated table from the given proxy, id and columns.
static BarrageTable	make(@Nullable String channelName, UpdateSourceRegistrar registrar, NotificationQueue queue, @Nullable ScheduledExecutorService executor, BarrageUtil.ConvertedArrowSchema schema, boolean isFullSubscription, @Nullable BarrageTable.ViewportChangedCallback vpCallback)
protected static @NotNull LinkedHashMap<String,ColumnSource<?>>	makeColumns(BarrageUtil.ConvertedArrowSchema schema, WritableColumnSource<?>[] writableSources)	Set up the columns for the replicated blink table.
protected static @NotNull LinkedHashMap<String,ColumnSource<?>>	makeColumns(BarrageUtil.ConvertedArrowSchema schema, WritableColumnSource<?>[] writableSources, WritableRowRedirection emptyRowRedirection)	Set up the columns for the replicated redirected table.
protected boolean	maybeEnablePrevTracking()
protected void	recordMetric(Function<BarrageTable.Stats,org.HdrHistogram.Histogram> hist, long value)
protected void	saveForDebugging(BarrageMessage snapshotOrDelta)
@NotNull Stream<PerformanceEntry>	sourceEntries()	Return a stream of performance entries that represent the InstrumentedTableUpdateSource which drive this table.
protected void	updateServerViewport(RowSet viewport, BitSet columns, boolean reverseViewport)

Methods inherited from class io.deephaven.engine.table.impl.QueryTable

aggAllBy, aggBy, aggNoMemo, apply, asOfJoin, byteColumnIterator, characterColumnIterator, checkInitiateBinaryOperation, checkInitiateOperation, columnIterator, copy, copy, copy, disableParallelWhereForThread, doubleColumnIterator, dropColumns, exactJoin, flatten, floatColumnIterator, getColumnSource, getColumnSourceMap, getColumnSources, getModifiedColumnSetForUpdates, getResult, getRowSet, getSubTable, getSubTable, head, headBy, headPct, integerColumnIterator, isFlat, join, lazyUpdate, longColumnIterator, memoizeResult, moveColumns, naturalJoin, newModifiedColumnSet, newModifiedColumnSetIdentityTransformer, newModifiedColumnSetIdentityTransformer, newModifiedColumnSetTransformer, newModifiedColumnSetTransformer, newModifiedColumnSetTransformer, newModifiedColumnSetTransformer, objectColumnIterator, objectColumnIterator, partitionBy, partitionedAggBy, propagateFlatness, rangeJoin, releaseCachedResources, renameColumns, reverse, rollup, select, selectDistinct, setFlat, setMemoizeResults, shortColumnIterator, silent, size, slice, slicePct, snapshot, snapshotWhen, sort, sort, tail, tailBy, tailPct, tree, ungroup, update, updateBy, updateView, validateSelect, view, where, whereIn, whereNotIn, withAdditionalColumns, withDefinitionUnsafe, wouldMatch

Methods inherited from class io.deephaven.engine.table.impl.BaseTable

addParentReference, addUpdateListener, addUpdateListener, addUpdateListener, append, assertAddOnly, assertAppendOnly, assertBlink, awaitUpdate, awaitUpdate, checkAvailableColumns, copyAttributes, copyAttributes, copySortableColumns, createSnapshotControlIfRefreshing, getDefinition, getLastNotificationStep, getUpdateGraph, hasListeners, initializeWithSnapshot, isAddOnly, isAppendOnly, isBlink, isFailed, isRefreshing, isSystemicObject, markSystemic, notifyListeners, notifyListeners, notifyListenersOnError, parentPerformanceEntryIds, removeBlink, removeUpdateListener, removeUpdateListener, satisfied, setLastNotificationStep, setRefreshing, setTotalsTable, shouldCopyAttribute, toString, withKeys, withUniqueKeys

Methods inherited from class io.deephaven.engine.table.impl.BaseGridAttributes

clearSortingRestrictions, getSortableColumns, restrictSortTo, setColumnDescriptions, setLayoutHints, setSortableColumns, withColumnDescription, withColumnDescriptions, withDescription

Methods inherited from class io.deephaven.engine.table.impl.LiveAttributeMap

copyAttributes, getAttributeKeys, getAttributes, getAttributes, hasAttribute, prepareReturnCopy, prepareReturnThis, published, retainingAttributes, setAttribute, setAttribute, withAttributes, withAttributes, withoutAttributes

Methods inherited from class io.deephaven.engine.liveness.LivenessArtifact

manageWithCurrentScope

Methods inherited from class io.deephaven.engine.liveness.ReferenceCountedLivenessNode

getWeakReference, initializeTransientFieldsForLiveness, onReferenceCountAtZero, tryManage, tryUnmanage, tryUnmanage

Methods inherited from class io.deephaven.engine.liveness.ReferenceCountedLivenessReferent

dropReference, tryRetainReference

Methods inherited from class io.deephaven.util.referencecounting.ReferenceCounted

decrementReferenceCount, forceReferenceCountToZero, getReferenceCountDebug, incrementReferenceCount, resetReferenceCount, tryDecrementReferenceCount, tryIncrementReferenceCount

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface io.deephaven.engine.table.AttributeMap

getAttributeKeys, getAttributes, getAttributes, hasAttribute, retainingAttributes, withAttributes, withAttributes, withoutAttributes

Methods inherited from interface io.deephaven.engine.table.GridAttributes

clearSortingRestrictions, restrictSortTo, setLayoutHints, withColumnDescription, withColumnDescriptions, withDescription

Methods inherited from interface io.deephaven.engine.liveness.LivenessManager

manage, tryManage, tryUnmanage, tryUnmanage, unmanage, unmanage

Methods inherited from interface io.deephaven.engine.liveness.LivenessReferent

dropReference, getReferentDescription, getWeakReference, retainReference, tryRetainReference

Methods inherited from interface io.deephaven.util.datastructures.LongSizedDataStructure

intSize, intSize

Methods inherited from interface io.deephaven.engine.updategraph.NotificationQueue.Dependency

getUpdateGraph

Methods inherited from interface io.deephaven.engine.table.impl.NotificationStepReceiver

initializeLastNotificationStep

Methods inherited from interface io.deephaven.engine.table.impl.TableDefaults

addUpdateListener, applyToAllBy, applyToAllBy, applyToAllBy, close, coalesce, dropColumnFormats, formatColumns, formatColumnWhere, formatRowWhere, getColumnSource, getColumnSource, hasColumns, hasColumns, headBy, isEmpty, join, meta, moveColumnsDown, moveColumnsUp, numColumns, partitionBy, renameAllColumns, renameColumns, rollup, rollup, rollup, rollup, rollup, selectDistinct, sizeForInstrumentation, snapshotWhen, snapshotWhen, tailBy, wouldMatch

Methods inherited from interface io.deephaven.api.TableOperationsDefaults

absSumBy, absSumBy, absSumBy, absSumBy, aggAllBy, aggAllBy, aggAllBy, aggBy, aggBy, aggBy, aggBy, aggBy, aggBy, aggBy, aj, aj, avgBy, avgBy, avgBy, avgBy, countBy, countBy, countBy, countBy, dropColumns, dropColumns, exactJoin, exactJoin, firstBy, firstBy, firstBy, firstBy, groupBy, groupBy, groupBy, join, join, join, lastBy, lastBy, lastBy, lastBy, lazyUpdate, maxBy, maxBy, maxBy, maxBy, medianBy, medianBy, medianBy, medianBy, minBy, minBy, minBy, minBy, naturalJoin, naturalJoin, naturalJoin, naturalJoin, naturalJoin, raj, raj, rangeJoin, select, select, selectDistinct, sort, sortDescending, stdBy, stdBy, stdBy, stdBy, sumBy, sumBy, sumBy, sumBy, ungroup, ungroup, ungroup, ungroup, update, updateBy, updateBy, updateBy, updateBy, updateBy, updateBy, updateBy, updateBy, updateView, varBy, varBy, varBy, varBy, view, wavgBy, wavgBy, wavgBy, wavgBy, where, whereIn, whereNotIn, wsumBy, wsumBy, wsumBy, wsumBy

Field Details

DEBUG_ENABLED

public static final boolean DEBUG_ENABLED

log

protected static final Logger log

BATCH_SIZE

protected static final int BATCH_SIZE

stats

protected final BarrageTable.Stats stats

capacity

protected long capacity

the capacity that the destSources been set to

destSources

protected final WritableColumnSource<?>[] destSources

the reinterpreted destination writable sources

Constructor Details

BarrageTable

protected BarrageTable(@Nullable
 @Nullable String channelName,
 UpdateSourceRegistrar registrar,
 NotificationQueue notificationQueue,
 @Nullable
 @Nullable ScheduledExecutorService executorService,
 LinkedHashMap<String,ColumnSource<?>> columns,
 WritableColumnSource<?>[] writableSources,
 Map<String,Object> attributes,
 @Nullable
 @Nullable BarrageTable.ViewportChangedCallback viewportChangedCallback)

Method Details

getDescription

public String getDescription()

Specified by:

getDescription in interface Table

Overrides:

getDescription in class BaseTable<QueryTable>

addSourceToRegistrar

public void addSourceToRegistrar()

Add this table to the registrar so that it can be refreshed.

ImplNote:

this cannot be performed in the constructor as the class is subclassed.

sourceEntries

@NotNull
public @NotNull Stream<PerformanceEntry> sourceEntries()

Description copied from interface: HasRefreshingSource

Return a stream of performance entries that represent the InstrumentedTableUpdateSource which drive this table.

If no such sources, exist, then an empty stream is returned.

Specified by:

sourceEntries in interface HasRefreshingSource

Returns:

a stream of performance entries for this node in the graph

applyUpdates

protected abstract TableUpdate applyUpdates(ArrayDeque<BarrageMessage> localPendingUpdates)

getWireChunkTypes

public ChunkType[] getWireChunkTypes()

getWireTypes

public Class<?>[] getWireTypes()

getWireComponentTypes

public Class<?>[] getWireComponentTypes()

getServerViewport

public RowSet getServerViewport()

getServerReverseViewport

public boolean getServerReverseViewport()

getServerColumns

public BitSet getServerColumns()

handleBarrageMessage

public void handleBarrageMessage(BarrageMessage update)

Specified by:

handleBarrageMessage in interface BarrageMessage.Listener

handleBarrageError

public void handleBarrageError(Throwable t)

Specified by:

handleBarrageError in interface BarrageMessage.Listener

updateServerViewport

protected void updateServerViewport(RowSet viewport,
 BitSet columns,
 boolean reverseViewport)

isSubscribedColumn

protected boolean isSubscribedColumn(int i)

getNotificationQueue

protected NotificationQueue getNotificationQueue()

Description copied from class: BaseTable

Get the notification queue to insert notifications into as they are generated by listeners during BaseTable.notifyListeners(io.deephaven.engine.rowset.RowSet, io.deephaven.engine.rowset.RowSet, io.deephaven.engine.rowset.RowSet) and BaseTable.notifyListenersOnError(Throwable, TableListener.Entry). This method may be overridden to provide a different notification queue than the table's PeriodicUpdateGraph instance for more complex behavior.

Overrides:

getNotificationQueue in class BaseTable<QueryTable>

Returns:

The NotificationQueue to add to

make

@InternalUseOnly
public static BarrageTable make(@Nullable
 @Nullable String channelName,
 @Nullable
 @Nullable ScheduledExecutorService executorService,
 @NotNull
 BarrageUtil.ConvertedArrowSchema schema,
 boolean isFullSubscription,
 @Nullable
 @Nullable BarrageTable.ViewportChangedCallback vpCallback)

Set up a replicated table from the given proxy, id and columns. This is intended for internal use only.

Parameters:

channelName - a string representation of the channel for use in Performance Logs

executorService - an executor service used to flush stats

schema - the table schema

isFullSubscription - whether this table is a full subscription

vpCallback - a callback for viewport changes

Returns:

a properly initialized BarrageTable

make

public static BarrageTable make(@Nullable
 @Nullable String channelName,
 UpdateSourceRegistrar registrar,
 NotificationQueue queue,
 @Nullable
 @Nullable ScheduledExecutorService executor,
 @NotNull
 BarrageUtil.ConvertedArrowSchema schema,
 boolean isFullSubscription,
 @Nullable
 @Nullable BarrageTable.ViewportChangedCallback vpCallback)

makeColumns

@NotNull
protected static @NotNull LinkedHashMap<String,ColumnSource<?>> makeColumns(BarrageUtil.ConvertedArrowSchema schema,
 WritableColumnSource<?>[] writableSources,
 WritableRowRedirection emptyRowRedirection)

Set up the columns for the replicated redirected table.

ApiNote:

emptyRowRedirection must be initialized and empty.

makeColumns

@NotNull
protected static @NotNull LinkedHashMap<String,ColumnSource<?>> makeColumns(BarrageUtil.ConvertedArrowSchema schema,
 WritableColumnSource<?>[] writableSources)

Set up the columns for the replicated blink table.

saveForDebugging

protected void saveForDebugging(BarrageMessage snapshotOrDelta)

maybeEnablePrevTracking

protected boolean maybeEnablePrevTracking()

doWakeup

protected void doWakeup()

getAttribute

@Nullable
public @Nullable Object getAttribute(@NotNull
 @NotNull String key)

Description copied from interface: AttributeMap

Get the value for the specified attribute key.

Specified by:

getAttribute in interface AttributeMap<Table>

Overrides:

getAttribute in class LiveAttributeMap<Table,QueryTable>

Parameters:

key - The name of the attribute

Returns:

The value, or null if there was none.

beginLog

protected LogEntry beginLog(LogLevel level)

Convenience method for writing consistent log messages from this object.

Parameters:

level - the log level

Returns:

a LogEntry

destroy

@OverridingMethodsMustInvokeSuper
protected void destroy()

Description copied from class: ReferenceCountedLivenessReferent

Attempt to release (destructively when necessary) resources held by this object. This may render the object unusable for subsequent operations. Implementations should be sure to call super.destroy().

This is intended to only ever be used as a side effect of decreasing the reference count to 0.

Overrides:

destroy in class BaseTable<QueryTable>

getDeserializationTmConsumer

public LongConsumer getDeserializationTmConsumer()

recordMetric

protected void recordMetric(Function<BarrageTable.Stats,org.HdrHistogram.Histogram> hist,
 long value)