Package io.deephaven.enterprise.locations.locationstable

Class LocationsTableUtil

java.lang.Object

io.deephaven.enterprise.locations.locationstable.LocationsTableUtil

public class LocationsTableUtil
extends Object

Provides utilities to assist in the creation and writing of a `LocationsTable`, which can be used to identify valid EnterpriseTableLocationKeys

Field Summary

Fields

Modifier and Type	Field	Description
static final String	COLUMN_VERSION_COLUMN_NAME	The column name containing the integer version of column files for Deephaven format.
static final String	FILENAME_COLUMN_NAME	The column of file names, relative to the root of this table.
static final String	FORMAT_COLUMN_NAME	The column name containing the format as a String value of TableLocation.Format (either "DEEPHAVEN" or "PARQUET").
static final String	LAST_MODIFIED_NANOS_COLUMN_NAME	The column name containing the epoch timestamp of the time the partition was last modified.
static final String	ROW_GROUP_SIZES_COLUMN_NAME	The column name sizes of each row group in parquet format.
static final String	SIZE_COLUMN_NAME	The column of partition sizes, in rows.

Constructor Summary

Constructors

Constructor	Description
LocationsTableUtil()

Method Summary

Modifier and Type	Method	Description
static TableDefinition	definitionWithPartitions(@NotNull Collection<ColumnDefinition<?>> partitioningCols)	Create a TableDefinition which includes all ColumnDefinitions for a new `LocationsTable`
static void	ensureValidLocationsDefinition(@NotNull TableDefinition locationsTableDef)	Ensures that all required columns of the appropriate type for a `LocationsTable` are included in the definition.
static void	ensureValidLocationsDefinition(@NotNull TableDefinition locationsTableDef, @NotNull Collection<ColumnDefinition<?>> partitioningCols)	Ensures that all required columns of the appropriate type for a `LocationsTable` are included in the definition.
static Table	generateLocationsTable(@NotNull PollingTableLocationProvider<EnterpriseTableKey,? extends EnterpriseTableLocationKey> pollingProvider, @NotNull List<ColumnDefinition<?>> columnPartitionKeys, @NotNull Path tableRoot)	Produce an in-memory table containing the files and relevant metadata for a given directory.
static void	writeLocationsTable(@NotNull Table locationsTable, @NotNull Path locationsDir)	Write the given locations table to disk at the defined path

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

FILENAME_COLUMN_NAME

public static final String FILENAME_COLUMN_NAME

The column of file names, relative to the root of this table.

See Also:

• Constant Field Values

SIZE_COLUMN_NAME

public static final String SIZE_COLUMN_NAME

The column of partition sizes, in rows.

See Also:

• Constant Field Values

FORMAT_COLUMN_NAME

public static final String FORMAT_COLUMN_NAME

The column name containing the format as a String value of TableLocation.Format (either "DEEPHAVEN" or "PARQUET").

See Also:

• Constant Field Values

LAST_MODIFIED_NANOS_COLUMN_NAME

public static final String LAST_MODIFIED_NANOS_COLUMN_NAME

The column name containing the epoch timestamp of the time the partition was last modified.

See Also:

• Constant Field Values

COLUMN_VERSION_COLUMN_NAME

public static final String COLUMN_VERSION_COLUMN_NAME

The column name containing the integer version of column files for Deephaven format.

See Also:

• Constant Field Values

ROW_GROUP_SIZES_COLUMN_NAME

public static final String ROW_GROUP_SIZES_COLUMN_NAME

The column name sizes of each row group in parquet format.

See Also:

• Constant Field Values

Constructor Details

LocationsTableUtil

public LocationsTableUtil()

Method Details

ensureValidLocationsDefinition

public static void ensureValidLocationsDefinition(@NotNull
 @NotNull TableDefinition locationsTableDef,
 @NotNull
 @NotNull Collection<ColumnDefinition<?>> partitioningCols)

Ensures that all required columns of the appropriate type for a `LocationsTable` are included in the definition.

Parameters:

locationsTableDef - the TableDefinition to check

partitioningCols - the partitioning Column

ensureValidLocationsDefinition

public static void ensureValidLocationsDefinition(@NotNull
 @NotNull TableDefinition locationsTableDef)

Ensures that all required columns of the appropriate type for a `LocationsTable` are included in the definition. Does not verify existence of partitioning columns or the existence of extra columns

Parameters:

locationsTableDef - the TableDefinition to check

definitionWithPartitions

public static TableDefinition definitionWithPartitions(@NotNull
 @NotNull Collection<ColumnDefinition<?>> partitioningCols)

Create a TableDefinition which includes all ColumnDefinitions for a new `LocationsTable`

Parameters:

partitioningCols - partitioning columns to define within the TableDefinition

Returns:

a TableDefinition which includes all columns for a `LocationsTable`

writeLocationsTable

public static void writeLocationsTable(@NotNull
 @NotNull Table locationsTable,
 @NotNull
 @NotNull Path locationsDir)
                                throws IOException

Write the given locations table to disk at the defined path

Parameters:

locationsTable - the Table to write to disk

locationsDir - the on-disk target for writing the Table

Throws:

IOException - if an error occurs while writing the locations table

generateLocationsTable

public static Table generateLocationsTable(@NotNull
 @NotNull PollingTableLocationProvider<EnterpriseTableKey,? extends EnterpriseTableLocationKey> pollingProvider,
 @NotNull
 @NotNull List<ColumnDefinition<?>> columnPartitionKeys,
 @NotNull
 @NotNull Path tableRoot)

Produce an in-memory table containing the files and relevant metadata for a given directory.

Parameters:

pollingProvider - provides all locations for the desired locations table

columnPartitionKeys - the partitioning colum(s)

tableRoot - the directory to scan

Returns:

an in-memory table suitable for use by writeLocationsTable(Table, Path)