Package io.deephaven.parquet.table

Class ParquetSchemaReader

java.lang.Object

io.deephaven.parquet.table.ParquetSchemaReader

public class ParquetSchemaReader
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	ParquetSchemaReader.ColumnDefinitionConsumer
static final class	ParquetSchemaReader.ParquetMessageDefinition

Constructor Summary

Constructors

Constructor	Description
ParquetSchemaReader()

Method Summary

Modifier and Type	Method	Description
static Pair<List<ColumnDefinition<?>>,ParquetInstructions>	convertSchema(@NotNull org.apache.parquet.schema.MessageType schema, @NotNull Map<String,String> keyValueMetadata, @NotNull ParquetInstructions readInstructionsIn)	Convert schema information from a ParquetMetadata into ColumnDefinitions.
static Optional<TableInfo>	parseMetadata(@NotNull Map<String,String> keyValueMetadata)
static ParquetInstructions	readParquetSchema(@NotNull org.apache.parquet.schema.MessageType schema, @NotNull Map<String,String> keyValueMetadata, @NotNull ParquetInstructions readInstructions, @NotNull ParquetSchemaReader.ColumnDefinitionConsumer consumer, @NotNull BiFunction<String,Set<String>,String> legalizeColumnNameFunc)	Obtain schema information from a parquet file

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ParquetSchemaReader

public ParquetSchemaReader()

Method Details

parseMetadata

public static Optional<TableInfo> parseMetadata(@NotNull
 @NotNull Map<String,String> keyValueMetadata)

readParquetSchema

public static ParquetInstructions readParquetSchema(@NotNull
 @NotNull org.apache.parquet.schema.MessageType schema,
 @NotNull
 @NotNull Map<String,String> keyValueMetadata,
 @NotNull
 @NotNull ParquetInstructions readInstructions,
 @NotNull
 @NotNull ParquetSchemaReader.ColumnDefinitionConsumer consumer,
 @NotNull
 @NotNull BiFunction<String,Set<String>,String> legalizeColumnNameFunc)

Obtain schema information from a parquet file

Parameters:

schema - Parquet schema

keyValueMetadata - Parquet key-value metadata map

readInstructions - Parquet read instructions specifying transformations like column mappings and codecs. Note a new read instructions based on this one may be returned by this method to provide necessary transformations, eg, replacing unsupported characters like ' ' (space) in column names.

consumer - A ColumnDefinitionConsumer whose accept method would be called for each column in the file

Returns:

Parquet read instructions, either the ones supplied or a new object based on the supplied with necessary transformations added.

convertSchema

public static Pair<List<ColumnDefinition<?>>,ParquetInstructions> convertSchema(@NotNull
 @NotNull org.apache.parquet.schema.MessageType schema,
 @NotNull
 @NotNull Map<String,String> keyValueMetadata,
 @NotNull
 @NotNull ParquetInstructions readInstructionsIn)

Convert schema information from a ParquetMetadata into ColumnDefinitions.

Parameters:

schema - Parquet schema

keyValueMetadata - Parquet key-value metadata map

readInstructionsIn - Input conversion ParquetInstructions

Returns:

A Pair with ColumnDefinitions and adjusted ParquetInstructions