Package io.deephaven.parquet.table.layout

Class ParquetMetadataFileLayout

java.lang.Object

io.deephaven.parquet.table.layout.ParquetMetadataFileLayout

All Implemented Interfaces:

TableLocationKeyFinder<ParquetTableLocationKey>

public class ParquetMetadataFileLayout
extends Object
implements TableLocationKeyFinder<ParquetTableLocationKey>

Location finder that will examine a parquet metadata file to discover locations.

Note that we expect to find the following files:

• _metadata - A file containing Parquet metadata for all row groups in all .parquet files for the entire data set, including schema information non-partitioning columns and key-value metadata
• _common_metadata (optional) - A file containing Parquet metadata with schema information that applies to the entire data set, including partitioning columns that are inferred from file paths rather than explicitly written in column chunks within .parquet files

Field Summary

Fields

Modifier and Type	Field	Description
static final String	COMMON_METADATA_FILE_NAME
static final String	METADATA_FILE_NAME

Constructor Summary

Constructors

Constructor	Description
ParquetMetadataFileLayout(@NotNull File directory)
ParquetMetadataFileLayout(@NotNull File directory, @NotNull ParquetInstructions inputInstructions)
ParquetMetadataFileLayout(@NotNull File metadataFile, @Nullable File commonMetadataFile)
ParquetMetadataFileLayout(@NotNull File metadataFile, @Nullable File commonMetadataFile, @NotNull ParquetInstructions inputInstructions)

Method Summary

Modifier and Type	Method	Description
void	findKeys(@NotNull Consumer<ParquetTableLocationKey> locationKeyObserver)	Find keys and deliver them to the locationKeyObserver.
ParquetInstructions	getInstructions()
TableDefinition	getTableDefinition()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

METADATA_FILE_NAME

public static final String METADATA_FILE_NAME

See Also:

• Constant Field Values

COMMON_METADATA_FILE_NAME

public static final String COMMON_METADATA_FILE_NAME

See Also:

• Constant Field Values

Constructor Details

ParquetMetadataFileLayout

public ParquetMetadataFileLayout(@NotNull
 @NotNull File directory)

ParquetMetadataFileLayout

public ParquetMetadataFileLayout(@NotNull
 @NotNull File directory,
 @NotNull
 @NotNull ParquetInstructions inputInstructions)

ParquetMetadataFileLayout

public ParquetMetadataFileLayout(@NotNull
 @NotNull File metadataFile,
 @Nullable
 @Nullable File commonMetadataFile)

ParquetMetadataFileLayout

public ParquetMetadataFileLayout(@NotNull
 @NotNull File metadataFile,
 @Nullable
 @Nullable File commonMetadataFile,
 @NotNull
 @NotNull ParquetInstructions inputInstructions)

Method Details

toString

public String toString()

Overrides:

toString in class Object

getTableDefinition

public TableDefinition getTableDefinition()

getInstructions

public ParquetInstructions getInstructions()

findKeys

public void findKeys(@NotNull
 @NotNull Consumer<ParquetTableLocationKey> locationKeyObserver)

Description copied from interface: TableLocationKeyFinder

Find keys and deliver them to the locationKeyObserver.

Specified by:

findKeys in interface TableLocationKeyFinder<ParquetTableLocationKey>

Parameters:

locationKeyObserver - Per-key callback