Package io.deephaven.parquet.table.location

Class ParquetTableLocation

java.lang.Object

io.deephaven.engine.table.impl.locations.impl.SubscriptionAggregator<TableLocation.Listener>

io.deephaven.engine.table.impl.locations.impl.AbstractTableLocation

io.deephaven.parquet.table.location.ParquetTableLocation

All Implemented Interfaces:

LogOutputAppendable, DelegatingLivenessReferent, LivenessReferent, TableLocation, TableLocationState, PushdownFilterMatcher, NamedImplementation

public class ParquetTableLocation
extends AbstractTableLocation

Nested Class Summary

Nested classes/interfaces inherited from interface io.deephaven.engine.table.impl.locations.TableLocation

TableLocation.Listener

Field Summary

Fields inherited from class io.deephaven.engine.table.impl.locations.impl.SubscriptionAggregator

subscriptions

Fields inherited from interface io.deephaven.engine.table.impl.locations.TableLocationState

NULL_SIZE, NULL_TIME

Constructor Summary

Constructors

Constructor	Description
ParquetTableLocation(@NotNull TableKey tableKey, @NotNull ParquetTableLocationKey tableLocationKey, @NotNull ParquetInstructions readInstructions)

Method Summary

Modifier and Type	Method	Description
void	estimatePushdownFilterCost(WhereFilter filter, RowSet selection, boolean usePrev, PushdownFilterContext context, JobScheduler jobScheduler, LongConsumer onComplete, Consumer<Exception> onError)	Estimate the cost of pushing down the next pushdown filter.
Map<String,ColumnTypeInfo>	getColumnTypes()
@NotNull List<String[]>	getDataIndexColumns()	Get a list of the columns by which this location is indexed
String	getImplementationName()	Get a name for the implementing class.
@NotNull List<SortColumn>	getSortedColumns()	Get an ordered list of columns this location is sorted by.
boolean	hasDataIndex(@NotNull String... columns)	Check if this TableLocation has a data index for the specified columns.
protected final void	initializeState()	No-op by default, can be overridden by subclasses to initialize state on first access.
@Nullable BasicDataIndex	loadDataIndex(@NotNull String... columns)	Load the data index from the location implementation.
protected @NotNull ColumnLocation	makeColumnLocation(@NotNull String columnName)
void	pushdownFilter(WhereFilter filter, RowSet selection, boolean usePrev, PushdownFilterContext context, long costCeiling, JobScheduler jobScheduler, Consumer<PushdownResult> onComplete, Consumer<Exception> onError)	Push down the given filter to the underlying table and pass the result to the consumer.
void	refresh()	Initialize or run state information.

Methods inherited from class io.deephaven.engine.table.impl.locations.impl.AbstractTableLocation

asLivenessReferent, clearColumnLocations, deliverInitialSnapshot, destroy, getColumnLocation, getDataIndex, getKey, getLastModifiedTimeMillis, getRowSet, getSize, getStateLock, getTableKey, handleUpdate, handleUpdate, handleUpdateInternal, handleUpdateInternal, hasAnyCachedDataIndex, hasCachedDataIndex, makePushdownFilterContext, toString

Methods inherited from class io.deephaven.engine.table.impl.locations.impl.SubscriptionAggregator

activateUnderlyingDataSource, activationFailed, activationSuccessful, checkHasSubscribers, deactivateUnderlyingDataSource, matchSubscriptionToken, postActivationHook, subscribe, supportsSubscriptions, unsubscribe

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface io.deephaven.engine.liveness.DelegatingLivenessReferent

dropReference, getWeakReference, tryRetainReference

Methods inherited from interface io.deephaven.engine.liveness.LivenessReferent

getReferentDescription, retainReference

Methods inherited from interface io.deephaven.engine.table.impl.locations.TableLocation

append, subscribe, supportsSubscriptions, toGenericString, toStringDetailed, toStringHelper, unsubscribe

Methods inherited from interface io.deephaven.engine.table.impl.locations.TableLocationState

copyStateValuesTo

Constructor Details

ParquetTableLocation

public ParquetTableLocation(@NotNull
 @NotNull TableKey tableKey,
 @NotNull
 @NotNull ParquetTableLocationKey tableLocationKey,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Method Details

getImplementationName

public String getImplementationName()

Description copied from interface: NamedImplementation

Get a name for the implementing class. Useful for abstract classes that implement LogOutputAppendable or override toString.

The default implementation is correct, but not suitable for high-frequency usage.

Returns:

A name for the implementing class

refresh

public void refresh()

Description copied from interface: TableLocation

Initialize or run state information.

getColumnTypes

public Map<String,ColumnTypeInfo> getColumnTypes()

getSortedColumns

@NotNull
public @NotNull List<SortColumn> getSortedColumns()

Description copied from interface: TableLocation

Get an ordered list of columns this location is sorted by.

Returns:

A non-null ordered list of SortColumns

initializeState

protected final void initializeState()

Description copied from class: AbstractTableLocation

No-op by default, can be overridden by subclasses to initialize state on first access.

The expectation for static locations that override this is to call AbstractTableLocation.handleUpdateInternal(RowSet, long) instead of AbstractTableLocation.handleUpdate(RowSet, long), and AbstractTableLocation.handleUpdateInternal(TableLocationState) instead of AbstractTableLocation.handleUpdate(TableLocationState) from inside AbstractTableLocation.initializeState(). Otherwise, the initialization logic will recurse infinitely.

Overrides:

initializeState in class AbstractTableLocation

makeColumnLocation

@NotNull
protected @NotNull ColumnLocation makeColumnLocation(@NotNull
 @NotNull String columnName)

Specified by:

makeColumnLocation in class AbstractTableLocation

getDataIndexColumns

@NotNull
public @NotNull List<String[]> getDataIndexColumns()

Description copied from interface: TableLocation

Get a list of the columns by which this location is indexed

Returns:

A non-null list of String[] arrays containing the key column names for each existing index

hasDataIndex

public boolean hasDataIndex(@NotNull
 @NotNull String... columns)

Description copied from interface: TableLocation

Check if this TableLocation has a data index for the specified columns.

Parameters:

columns - The set of columns to check for

Returns:

Whether the TableLocation has an index for the specified columns

loadDataIndex

@Nullable
public @Nullable BasicDataIndex loadDataIndex(@NotNull
 @NotNull String... columns)

Description copied from class: AbstractTableLocation

Load the data index from the location implementation. Implementations of this method should not perform any result caching.

Specified by:

loadDataIndex in class AbstractTableLocation

Parameters:

columns - The columns to load an index for

Returns:

The data index, or null if none exists

estimatePushdownFilterCost

public void estimatePushdownFilterCost(WhereFilter filter,
 RowSet selection,
 boolean usePrev,
 PushdownFilterContext context,
 JobScheduler jobScheduler,
 LongConsumer onComplete,
 Consumer<Exception> onError)

Description copied from interface: PushdownFilterMatcher

Estimate the cost of pushing down the next pushdown filter. This returns a unitless value to compare the cost of executing different filters. Common costs are listed in PushdownResult (such as PushdownResult.METADATA_STATS_COST) and should be used as a baseline for estimating the cost of newly implemented pushdown operations.

A no-op implementation should simply complete with Long.MAX_VALUE.

Specified by:

estimatePushdownFilterCost in interface PushdownFilterMatcher

Overrides:

estimatePushdownFilterCost in class AbstractTableLocation

Parameters:

filter - The filter to test.

selection - The set of rows to tests.

usePrev - Whether to use the previous result

context - The PushdownFilterContext to use for the pushdown operation.

jobScheduler - The job scheduler to use for scheduling child jobs

onComplete - Consumer of the estimated cost of the pushdown operation. 9223372036854775807L indicates that the filter cannot be pushed down.

onError - Consumer of any exceptions that occur during the estimate operation

pushdownFilter

public void pushdownFilter(WhereFilter filter,
 RowSet selection,
 boolean usePrev,
 PushdownFilterContext context,
 long costCeiling,
 JobScheduler jobScheduler,
 Consumer<PushdownResult> onComplete,
 Consumer<Exception> onError)

Description copied from interface: PushdownFilterMatcher

Push down the given filter to the underlying table and pass the result to the consumer. This method is expected to execute all pushdown filter steps that are greater than PushdownFilterContext.executedFilterCost() and less than or equal to costCeiling.

The resulting PushdownResult (to onComplete) must only contain rows from selection. The match row set are rows that are guaranteed to match. The implicitly "missing" rows selection - match - maybeMatch are rows that are guaranteed to not match. The remaining maybe match row set are rows that may, or may not, match. The pushdown result ownership passes to onComplete.

A no-op implementation should simply complete with PushdownResult.maybeMatch(selection).

Specified by:

pushdownFilter in interface PushdownFilterMatcher

Overrides:

pushdownFilter in class AbstractTableLocation

Parameters:

filter - The filter to apply.

selection - The set of rows to test.

usePrev - Whether to use the previous result

context - The PushdownFilterContext to use for the pushdown operation.

costCeiling - Execute all possible filters with a cost less than or equal this value.

jobScheduler - The job scheduler to use for scheduling child jobs

onComplete - Consumer of the output rowsets for added and modified rows that pass the filter

onError - Consumer of any exceptions that occur during the pushdown operation