Package io.deephaven.parquet.base

Class ParquetFileReader

java.lang.Object

io.deephaven.parquet.base.ParquetFileReader

public class ParquetFileReader
extends Object

Top level accessor for a parquet file which can read both from a file path string or a CLI style file URI, ex."s3://bucket/key".

Field Summary

Fields

Modifier and Type	Field	Description
static final String	FILE_URI_SCHEME
final org.apache.parquet.format.FileMetaData	fileMetaData

Constructor Summary

Constructors

Constructor	Description
ParquetFileReader(String source, SeekableChannelsProvider channelsProvider)	Create a new ParquetFileReader for the provided source.
ParquetFileReader(URI parquetFileURI, SeekableChannelsProvider channelsProvider)	Create a new ParquetFileReader for the provided source.

Method Summary

Modifier and Type	Method	Description
static ParquetFileReader	create(@NotNull File parquetFile, @Nullable Object specialInstructions)	Make a ParquetFileReader for the supplied File.
static ParquetFileReader	create(@NotNull URI parquetFileURI, @Nullable Object specialInstructions)	Make a ParquetFileReader for the supplied URI.
static ParquetFileReader	createChecked(@NotNull File parquetFile, @Nullable Object specialInstructions)	Make a ParquetFileReader for the supplied File.
static ParquetFileReader	createChecked(@NotNull URI parquetFileURI, @Nullable Object specialInstructions)	Make a ParquetFileReader for the supplied URI.
SeekableChannelsProvider	getChannelsProvider()
Set<String>	getColumnsWithDictionaryUsedOnEveryDataPage()	Get the name of all columns that we can know for certain (a) have a dictionary, and (b) use the dictionary on all data pages.
RowGroupReader	getRowGroup(int groupNumber, String version)	Create a RowGroupReader object for provided row group number
org.apache.parquet.schema.MessageType	getSchema()
int	rowGroupCount()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

FILE_URI_SCHEME

public static final String FILE_URI_SCHEME

See Also:

• Constant Field Values

fileMetaData

public final org.apache.parquet.format.FileMetaData fileMetaData

Constructor Details

ParquetFileReader

public ParquetFileReader(String source,
 SeekableChannelsProvider channelsProvider)
                  throws IOException

Create a new ParquetFileReader for the provided source.

Parameters:

source - The source path or URI for the parquet file or the parquet metadata file

channelsProvider - The SeekableChannelsProvider to use for reading the file

Throws:

IOException

ParquetFileReader

public ParquetFileReader(URI parquetFileURI,
 SeekableChannelsProvider channelsProvider)
                  throws IOException

Create a new ParquetFileReader for the provided source.

Parameters:

parquetFileURI - The URI for the parquet file or the parquet metadata file

channelsProvider - The SeekableChannelsProvider to use for reading the file

Throws:

IOException

Method Details

create

public static ParquetFileReader create(@NotNull
 @NotNull File parquetFile,
 @Nullable
 @Nullable Object specialInstructions)

Make a ParquetFileReader for the supplied File. Wraps IOException as UncheckedDeephavenException.

Parameters:

parquetFile - The parquet file or the parquet metadata file

specialInstructions - Optional read instructions to pass to SeekableChannelsProvider while creating channels

Returns:

The new ParquetFileReader

create

public static ParquetFileReader create(@NotNull
 @NotNull URI parquetFileURI,
 @Nullable
 @Nullable Object specialInstructions)

Make a ParquetFileReader for the supplied URI. Wraps IOException as UncheckedDeephavenException.

Parameters:

parquetFileURI - The URI for the parquet file or the parquet metadata file

specialInstructions - Optional read instructions to pass to SeekableChannelsProvider while creating channels

Returns:

The new ParquetFileReader

createChecked

public static ParquetFileReader createChecked(@NotNull
 @NotNull File parquetFile,
 @Nullable
 @Nullable Object specialInstructions)
                                       throws IOException

Make a ParquetFileReader for the supplied File.

Parameters:

parquetFile - The parquet file or the parquet metadata file

specialInstructions - Optional read instructions to pass to SeekableChannelsProvider while creating channels

Returns:

The new ParquetFileReader

Throws:

IOException - if an IO exception occurs

createChecked

public static ParquetFileReader createChecked(@NotNull
 @NotNull URI parquetFileURI,
 @Nullable
 @Nullable Object specialInstructions)
                                       throws IOException

Make a ParquetFileReader for the supplied URI.

Parameters:

parquetFileURI - The URI for the parquet file or the parquet metadata file

specialInstructions - Optional read instructions to pass to SeekableChannelsProvider while creating channels

Returns:

The new ParquetFileReader

Throws:

IOException - if an IO exception occurs

getChannelsProvider

public SeekableChannelsProvider getChannelsProvider()

Returns:

The SeekableChannelsProvider used for this reader, appropriate to use for related file access

getColumnsWithDictionaryUsedOnEveryDataPage

public Set<String> getColumnsWithDictionaryUsedOnEveryDataPage()

Get the name of all columns that we can know for certain (a) have a dictionary, and (b) use the dictionary on all data pages.

Returns:

A set of parquet column names that satisfies the required condition.

getRowGroup

public RowGroupReader getRowGroup(int groupNumber,
 String version)

Create a RowGroupReader object for provided row group number

Parameters:

version - The "version" string from deephaven specific parquet metadata, or null if it's not present.

getSchema

public org.apache.parquet.schema.MessageType getSchema()

rowGroupCount

public int rowGroupCount()