Package com.illumon.iris.db.v2.locations.parquet

Class ParquetSchemaReader

java.lang.Object

com.illumon.iris.db.v2.locations.parquet.ParquetSchemaReader

public class ParquetSchemaReader
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	ParquetSchemaReader.ColumnDefinitionConsumer
static final class	ParquetSchemaReader.ParquetMessageDefinition

Constructor Summary

Constructors

Constructor	Description
ParquetSchemaReader()

Method Summary

Modifier and Type	Method	Description
static Optional<TableInfo>	parseMetadata(Map<String,String> keyValueMetadata)
static ParquetInstructions	readParquetSchema(org.apache.parquet.schema.MessageType schema, Map<String,String> keyValueMetadata, ParquetInstructions readInstructions, ParquetSchemaReader.ColumnDefinitionConsumer consumer, BiFunction<String,Set<String>,String> legalizeColumnNameFunc)	Obtain schema information from a parquet file

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ParquetSchemaReader

public ParquetSchemaReader()

Method Details

parseMetadata

public static Optional<TableInfo> parseMetadata(@NotNull
 Map<String,String> keyValueMetadata)

readParquetSchema

public static ParquetInstructions readParquetSchema(@NotNull
 org.apache.parquet.schema.MessageType schema,
 @NotNull
 Map<String,String> keyValueMetadata,
 @NotNull
 ParquetInstructions readInstructions,
 @NotNull
 ParquetSchemaReader.ColumnDefinitionConsumer consumer,
 @NotNull
 BiFunction<String,Set<String>,String> legalizeColumnNameFunc)

Obtain schema information from a parquet file

Parameters:

schema - Parquet schema. DO NOT RELY ON ParquetMetadataConverter FOR THIS! USE ParquetFileReader!

keyValueMetadata - Parquet key-value metadata map

readInstructions - Parquet read instructions specifying transformations like column mappings and codecs. Note a new read instructions based on this one may be returned by this method to provide necessary transformations, eg, replacing unsupported characters like ' ' (space) in column names.

consumer - A ColumnDefinitionConsumer whose accept method would be called for each column in the file

Returns:

Parquet read instructions, either the ones supplied or a new object based on the supplied with necessary transformations added.