Package io.deephaven.parquet.table.location

Class ParquetTableLocation

java.lang.Object

io.deephaven.engine.table.impl.locations.impl.SubscriptionAggregator<TableLocation.Listener>

io.deephaven.engine.table.impl.locations.impl.AbstractTableLocation

io.deephaven.parquet.table.location.ParquetTableLocation

All Implemented Interfaces:

LogOutputAppendable, DelegatingLivenessReferent, LivenessReferent, TableLocation, TableLocationState, PushdownFilterMatcher, RegionedPushdownFilterMatcher, NamedImplementation

public class ParquetTableLocation
extends AbstractTableLocation

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	ParquetTableLocation.ActionContext
static class	ParquetTableLocation.EstimateContext

Nested classes/interfaces inherited from interface io.deephaven.engine.table.impl.locations.TableLocation

TableLocation.Listener

Field Summary

Fields inherited from class io.deephaven.engine.table.impl.locations.impl.SubscriptionAggregator

subscriptions

Fields inherited from interface io.deephaven.engine.table.impl.locations.TableLocationState

NULL_SIZE, NULL_TIME

Constructor Summary

Constructors

Constructor	Description
ParquetTableLocation(@NotNull TableKey tableKey, @NotNull ParquetTableLocationKey tableLocationKey, @NotNull ParquetInstructions readInstructions)

Method Summary

Modifier and Type	Method	Description
long	estimatePushdownAction(RegionedPushdownAction action, WhereFilter filter, RowSet selection, boolean usePrev, PushdownFilterContext filterContext, RegionedPushdownAction.EstimateContext estimateContext)	Estimate the cost of the given pushdown action.
Map<String,ColumnTypeInfo>	getColumnTypes()
@NotNull List<String[]>	getDataIndexColumns()	Get a list of the columns by which this location is indexed
String	getImplementationName()	Get a name for the implementing class.
@NotNull List<SortColumn>	getSortedColumns()	Get an ordered list of columns this location is sorted by.
boolean	hasDataIndex(@NotNull String... columns)	Check if this TableLocation has a data index for the specified columns.
protected final void	initializeState()	No-op by default, can be overridden by subclasses to initialize state on first access.
@Nullable BasicDataIndex	loadDataIndex(@NotNull String... columns)	Load the data index from the location implementation.
RegionedPushdownAction.ActionContext	makeActionContext(WhereFilter filter, PushdownFilterContext filterContext)	Create a context for processing the given filter and filter context.
protected @NotNull ColumnLocation	makeColumnLocation(@NotNull String columnName)
RegionedPushdownAction.EstimateContext	makeEstimateContext(WhereFilter filter, PushdownFilterContext filterContext)	Create a context for estimating the cost for the filter and filter context.
PushdownResult	performPushdownAction(RegionedPushdownAction action, WhereFilter filter, RowSet selection, PushdownResult input, boolean usePrev, PushdownFilterContext filterContext, RegionedPushdownAction.ActionContext actionContext)	Perform the pushdown action for the given filter and filter context.
static @NotNull PushdownResult	pushdownDataIndex(RowSet selection, WhereFilter filter, Map<String,String> renameMap, BasicDataIndex dataIndex, PushdownResult result)	Apply the filter to the data index table and return the result.
void	refresh()	Initialize or run state information.
List<RegionedPushdownAction>	supportedActions()	Get the list of pushdown actions supported by this matcher.

Methods inherited from class io.deephaven.engine.table.impl.locations.impl.AbstractTableLocation

asLivenessReferent, clearColumnLocations, deliverInitialSnapshot, destroy, estimatePushdownFilterCost, getColumnLocation, getDataIndex, getKey, getLastModifiedTimeMillis, getRowSet, getSize, getStateLock, getTableKey, handleUpdate, handleUpdate, handleUpdateInternal, handleUpdateInternal, hasAnyCachedDataIndex, hasCachedDataIndex, makePushdownFilterContext, pushdownFilter, toString

Methods inherited from class io.deephaven.engine.table.impl.locations.impl.SubscriptionAggregator

activateUnderlyingDataSource, activationFailed, activationSuccessful, checkHasSubscribers, deactivateUnderlyingDataSource, matchSubscriptionToken, postActivationHook, subscribe, supportsSubscriptions, unsubscribe

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface io.deephaven.engine.liveness.DelegatingLivenessReferent

dropReference, getWeakReference, tryRetainReference

Methods inherited from interface io.deephaven.engine.liveness.LivenessReferent

getReferentDescription, retainReference

Methods inherited from interface io.deephaven.engine.table.impl.locations.TableLocation

append, subscribe, supportsSubscriptions, toGenericString, toStringDetailed, toStringHelper, unsubscribe

Methods inherited from interface io.deephaven.engine.table.impl.locations.TableLocationState

copyStateValuesTo

Constructor Details

ParquetTableLocation

public ParquetTableLocation(@NotNull
 @NotNull TableKey tableKey,
 @NotNull
 @NotNull ParquetTableLocationKey tableLocationKey,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Method Details

getImplementationName

public String getImplementationName()

Description copied from interface: NamedImplementation

Get a name for the implementing class. Useful for abstract classes that implement LogOutputAppendable or override toString.

The default implementation is correct, but not suitable for high-frequency usage.

Returns:

A name for the implementing class

refresh

public void refresh()

Description copied from interface: TableLocation

Initialize or run state information.

getColumnTypes

public Map<String,ColumnTypeInfo> getColumnTypes()

getSortedColumns

@NotNull
public @NotNull List<SortColumn> getSortedColumns()

Description copied from interface: TableLocation

Get an ordered list of columns this location is sorted by.

Returns:

A non-null ordered list of SortColumns

initializeState

protected final void initializeState()

Description copied from class: AbstractTableLocation

No-op by default, can be overridden by subclasses to initialize state on first access.

The expectation for static locations that override this is to call AbstractTableLocation.handleUpdateInternal(RowSet, long) instead of AbstractTableLocation.handleUpdate(RowSet, long), and AbstractTableLocation.handleUpdateInternal(TableLocationState) instead of AbstractTableLocation.handleUpdate(TableLocationState) from inside AbstractTableLocation.initializeState(). Otherwise, the initialization logic will recurse infinitely.

Overrides:

initializeState in class AbstractTableLocation

makeColumnLocation

@NotNull
protected @NotNull ColumnLocation makeColumnLocation(@NotNull
 @NotNull String columnName)

Specified by:

makeColumnLocation in class AbstractTableLocation

getDataIndexColumns

@NotNull
public @NotNull List<String[]> getDataIndexColumns()

Description copied from interface: TableLocation

Get a list of the columns by which this location is indexed

Returns:

A non-null list of String[] arrays containing the key column names for each existing index

hasDataIndex

public boolean hasDataIndex(@NotNull
 @NotNull String... columns)

Check if this TableLocation has a data index for the specified columns.

An index is trusted when this location's own file declares it and the index file exists; nothing else about the index is checked against the file it indexes, and pushdownDataIndex(io.deephaven.engine.rowset.RowSet, io.deephaven.engine.table.impl.select.WhereFilter, java.util.Map<java.lang.String, java.lang.String>, io.deephaven.engine.table.BasicDataIndex, io.deephaven.engine.table.impl.PushdownResult) treats every row the index does not place under a matching key as not matching. Every index a file declares must therefore have been written for that file: a writer that replaces the file, or an index it declares, without the other makes filters silently drop rows. An index file that no current file declares, such as one left behind when a file is rewritten without indexes, is ignored. Deephaven's own writers commit a file and the indexes it declares together.

Parameters:

columns - The set of columns to check for

Returns:

Whether the TableLocation has an index for the specified columns

loadDataIndex

@Nullable
public @Nullable BasicDataIndex loadDataIndex(@NotNull
 @NotNull String... columns)

Description copied from class: AbstractTableLocation

Load the data index from the location implementation. Implementations of this method should not perform any result caching.

Specified by:

loadDataIndex in class AbstractTableLocation

Parameters:

columns - The columns to load an index for

Returns:

The data index, or null if none exists

supportedActions

public List<RegionedPushdownAction> supportedActions()

Description copied from interface: RegionedPushdownFilterMatcher

Get the list of pushdown actions supported by this matcher. Defaults to an empty list.

makeEstimateContext

public RegionedPushdownAction.EstimateContext makeEstimateContext(WhereFilter filter,
 PushdownFilterContext filterContext)

Description copied from interface: RegionedPushdownFilterMatcher

Create a context for estimating the cost for the filter and filter context. This context can be used to provide additional information to the RegionedPushdownFilterMatcher.estimatePushdownAction(io.deephaven.engine.table.impl.sources.regioned.RegionedPushdownAction, io.deephaven.engine.table.impl.select.WhereFilter, io.deephaven.engine.rowset.RowSet, boolean, io.deephaven.engine.table.impl.PushdownFilterContext, io.deephaven.engine.table.impl.sources.regioned.RegionedPushdownAction.EstimateContext) function

estimatePushdownAction

public long estimatePushdownAction(RegionedPushdownAction action,
 WhereFilter filter,
 RowSet selection,
 boolean usePrev,
 PushdownFilterContext filterContext,
 RegionedPushdownAction.EstimateContext estimateContext)

Description copied from interface: RegionedPushdownFilterMatcher

Estimate the cost of the given pushdown action. Returns Long.MAX_VALUE if the action is not supported by this matcher.

makeActionContext

public RegionedPushdownAction.ActionContext makeActionContext(WhereFilter filter,
 PushdownFilterContext filterContext)

Description copied from interface: RegionedPushdownFilterMatcher

Create a context for processing the given filter and filter context. This context can be used to provide additional information to the RegionedPushdownFilterMatcher.performPushdownAction(io.deephaven.engine.table.impl.sources.regioned.RegionedPushdownAction, io.deephaven.engine.table.impl.select.WhereFilter, io.deephaven.engine.rowset.RowSet, io.deephaven.engine.table.impl.PushdownResult, boolean, io.deephaven.engine.table.impl.PushdownFilterContext, io.deephaven.engine.table.impl.sources.regioned.RegionedPushdownAction.ActionContext) function

performPushdownAction

public PushdownResult performPushdownAction(RegionedPushdownAction action,
 WhereFilter filter,
 RowSet selection,
 PushdownResult input,
 boolean usePrev,
 PushdownFilterContext filterContext,
 RegionedPushdownAction.ActionContext actionContext)

Description copied from interface: RegionedPushdownFilterMatcher

Perform the pushdown action for the given filter and filter context.

pushdownDataIndex

@NotNull
public static @NotNull PushdownResult pushdownDataIndex(RowSet selection,
 WhereFilter filter,
 Map<String,String> renameMap,
 BasicDataIndex dataIndex,
 PushdownResult result)

Apply the filter to the data index table and return the result. When the index is applied, the result has no maybe matches: rows the index does not place under a matching key are treated as not matching, so the index must be complete and current for the rows it covers (see hasDataIndex(java.lang.String...)). When the selection is too small for the index, or filtering the index fails for a reason other than cancellation, the result is a copy of result.