Package io.deephaven.parquet.table

Class ParquetTools

java.lang.Object

io.deephaven.parquet.table.ParquetTools

public class ParquetTools
extends Object

Tools for managing and manipulating tables on disk in parquet format.

Field Summary

Fields

Modifier and Type	Field	Description
static final ParquetInstructions	BROTLI
static final ParquetInstructions	GZIP
static final ParquetInstructions	LEGACY
static final ParquetInstructions	LZ4	Deprecated. Use LZ4_RAW instead, as explained here
static final ParquetInstructions	LZ4_RAW
static final ParquetInstructions	LZO
static final ParquetInstructions	SNAPPY
static final ParquetInstructions	UNCOMPRESSED
static final ParquetInstructions	ZSTD

Method Summary

Modifier and Type	Method	Description
static void	deleteTable(File path)	Deletes a table on disk.
static String	legacyGroupingFileName(@NotNull File tableDest, @NotNull String columnName)	Legacy method for generating a grouping file name.
static Table	readFlatPartitionedTable(@NotNull File directory, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.FLAT_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).
static Table	readFlatPartitionedTable(@NotNull File directory, @NotNull ParquetInstructions readInstructions, @NotNull TableDefinition tableDefinition)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.FLAT_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static Table	readKeyValuePartitionedTable(@NotNull File directory, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.KV_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).
static Table	readKeyValuePartitionedTable(@NotNull File directory, @NotNull ParquetInstructions readInstructions, @NotNull TableDefinition tableDefinition)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.KV_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static Table	readParquetSchemaAndTable(@NotNull File source, @NotNull ParquetInstructions readInstructionsIn, @Nullable org.apache.commons.lang3.mutable.MutableObject<ParquetInstructions> mutableInstructionsOut)
static Table	readPartitionedTable(@NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder, @NotNull ParquetInstructions readInstructions)	Deprecated. use readTable(TableLocationKeyFinder, ParquetInstructions) instead
static Table	readPartitionedTable(@NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder, @NotNull ParquetInstructions readInstructions, @NotNull TableDefinition tableDefinition)	Deprecated. use readTable(TableLocationKeyFinder, ParquetInstructions) instead with the table definition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static Table	readPartitionedTableInferSchema(@NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder, @NotNull ParquetInstructions readInstructions)	Deprecated. use readTable(TableLocationKeyFinder, ParquetInstructions)
static Table	readPartitionedTableWithMetadata(@NotNull File directory, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.METADATA_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).
static Table	readPartitionedTableWithMetadata(@NotNull String directory, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.METADATA_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).
static Table	readSingleFileTable(@NotNull ParquetTableLocationKey tableLocationKey, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).
static Table	readSingleFileTable(@NotNull ParquetTableLocationKey tableLocationKey, @NotNull ParquetInstructions readInstructions, @NotNull TableDefinition tableDefinition)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition)
static Table	readSingleFileTable(@NotNull File file, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).
static Table	readSingleFileTable(@NotNull File file, @NotNull ParquetInstructions readInstructions, @NotNull TableDefinition tableDefinition)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static Table	readSingleFileTable(@NotNull String source, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).
static Table	readSingleFileTable(@NotNull String source, @NotNull ParquetInstructions readInstructions, @NotNull TableDefinition tableDefinition)	Deprecated. Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static Table	readTable(@NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder, @NotNull ParquetInstructions readInstructions)	Reads in a table from files discovered with locationKeyFinder using a definition either provided using ParquetInstructions or built from the highest (by location key order) location found, which must have non-null partition values for all partition keys.
static Table	readTable(@NotNull File sourceFile)	Deprecated. Use readTable(String) instead.
static Table	readTable(@NotNull File sourceFile, @NotNull ParquetInstructions readInstructions)	Deprecated. Use readTable(String, ParquetInstructions) instead.
static Table	readTable(@NotNull String source)	Reads in a table from a single parquet file, metadata file, or directory with recognized layout.
static Table	readTable(@NotNull String source, @NotNull ParquetInstructions readInstructions)	Reads in a table from a single parquet file, metadata file, or directory with recognized layout.
static void	setDefaultCompressionCodecName(String compressionCodecName)	Deprecated. Do not use this method, instead pass the above codecs as arguments to writeTable(Table, String, ParquetInstructions) method
static void	writeKeyValuePartitionedTable(@NotNull PartitionedTable partitionedTable, @NotNull String destinationDir, @NotNull ParquetInstructions writeInstructions)	Write a partitioned table to disk in parquet format with all the key columns as "key=value" format in a nested directory structure.
static void	writeKeyValuePartitionedTable(@NotNull Table sourceTable, @NotNull String destinationDir, @NotNull ParquetInstructions writeInstructions)	Write table to disk in parquet format with partitioning columns written as "key=value" format in a nested directory structure.
static void	writeTable(@NotNull Table sourceTable, @NotNull File destFile)	Deprecated. Use writeTable(Table, String) instead.
static void	writeTable(@NotNull Table sourceTable, @NotNull File destFile, @NotNull TableDefinition definition)	Deprecated. Use writeTable(Table, String, ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static void	writeTable(@NotNull Table sourceTable, @NotNull File destFile, @NotNull TableDefinition definition, @NotNull ParquetInstructions writeInstructions)	Deprecated. Use writeTable(Table, String, ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static void	writeTable(@NotNull Table sourceTable, @NotNull File destFile, @NotNull ParquetInstructions writeInstructions)	Deprecated. Use writeTable(Table, String, ParquetInstructions) instead.
static void	writeTable(@NotNull Table sourceTable, @NotNull String destination)	Write a table to a file.
static void	writeTable(@NotNull Table sourceTable, @NotNull String destination, @NotNull TableDefinition definition, @NotNull ParquetInstructions writeInstructions)	Deprecated. Use writeTable(Table, String, ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static void	writeTable(@NotNull Table sourceTable, @NotNull String destination, @NotNull ParquetInstructions writeInstructions)	Write a table to a file.
static void	writeTables(@NotNull Table[] sources, @NotNull TableDefinition definition, @NotNull File[] destinations)	Deprecated. Use writeTables(Table[], String[], ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static void	writeTables(@NotNull Table[] sources, @NotNull TableDefinition definition, @NotNull File[] destinations, @NotNull ParquetInstructions writeInstructions)	Deprecated. Use writeTables(Table[], String[], ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).
static void	writeTables(@NotNull Table[] sources, @NotNull String[] destinations, @NotNull ParquetInstructions writeInstructions)	Write out tables to disk.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

UNCOMPRESSED

public static final ParquetInstructions UNCOMPRESSED

LZ4

@Deprecated
public static final ParquetInstructions LZ4

Deprecated.

Use LZ4_RAW instead, as explained here

LZ4_RAW

public static final ParquetInstructions LZ4_RAW

LZO

public static final ParquetInstructions LZO

GZIP

public static final ParquetInstructions GZIP

ZSTD

public static final ParquetInstructions ZSTD

SNAPPY

public static final ParquetInstructions SNAPPY

BROTLI

public static final ParquetInstructions BROTLI

LEGACY

public static final ParquetInstructions LEGACY

Method Details

readTable

public static Table readTable(@NotNull
 @NotNull String source)

Reads in a table from a single parquet file, metadata file, or directory with recognized layout. The source provided can be a local file path or a URI to be resolved via the provided SeekableChannelsProviderPlugin.

This method attempts to "do the right thing." It examines the source to determine if it's a single parquet file, a metadata file, or a directory. If it's a directory, it additionally tries to guess the layout to use. Unless a metadata file is supplied or discovered in the directory, the highest (by location key order) location found will be used to infer schema.

Parameters:

source - The path or URI of file or directory to examine

Returns:

table

See Also:

• ParquetMetadataFileLayout
• ParquetKeyValuePartitionedLayout
• ParquetFlatPartitionedLayout

readTable

public static Table readTable(@NotNull
 @NotNull String source,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Reads in a table from a single parquet file, metadata file, or directory with recognized layout. The source provided can be a local file path or a URI to be resolved via the provided SeekableChannelsProviderPlugin.

If the ParquetInstructions.ParquetFileLayout is not provided in the instructions, this method attempts to "do the right thing." It examines the source to determine if it's a single parquet file, a metadata file, or a directory. If it's a directory, it additionally tries to guess the layout to use. Unless a metadata file is supplied or discovered in the directory, the highest (by location key order) location found will be used to infer schema.

Parameters:

source - The path or URI of file or directory to examine

readInstructions - Instructions for customizations while reading

Returns:

table

See Also:

• ParquetMetadataFileLayout
• ParquetKeyValuePartitionedLayout
• ParquetFlatPartitionedLayout

readTable

@Deprecated
public static Table readTable(@NotNull
 @NotNull File sourceFile)

Deprecated.

Use readTable(String) instead.

Reads in a table from a single parquet file, metadata file, or directory with recognized layout.

This method attempts to "do the right thing." It examines the source to determine if it's a single parquet file, a metadata file, or a directory. If it's a directory, it additionally tries to guess the layout to use. Unless a metadata file is supplied or discovered in the directory, the highest (by location key order) location found will be used to infer schema.

Parameters:

sourceFile - The file or directory to examine

Returns:

table

See Also:

• ParquetMetadataFileLayout
• ParquetKeyValuePartitionedLayout
• ParquetFlatPartitionedLayout

readTable

@Deprecated
public static Table readTable(@NotNull
 @NotNull File sourceFile,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead.

Reads in a table from a single parquet file, metadata file, or directory with recognized layout.

This method attempts to "do the right thing." It examines the source to determine if it's a single parquet file, a metadata file, or a directory. If it's a directory, it additionally tries to guess the layout to use. Unless a metadata file is supplied or discovered in the directory, the highest (by location key order) location found will be used to infer schema.

Delegates to one of readSingleFileTable(File, ParquetInstructions), readPartitionedTableWithMetadata(File, ParquetInstructions), readFlatPartitionedTable(File, ParquetInstructions), or readKeyValuePartitionedTable(File, ParquetInstructions).

Parameters:

sourceFile - The file or directory to examine

readInstructions - Instructions for customizations while reading

Returns:

table

See Also:

• ParquetMetadataFileLayout
• ParquetKeyValuePartitionedLayout
• ParquetFlatPartitionedLayout

writeTable

public static void writeTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull String destination)

Write a table to a file. Data indexes to write are determined by those present on sourceTable.

Parameters:

sourceTable - source table

destination - destination path or URI; the file name should end in ".parquet" extension. If the path includes non-existing directories, they are created. If there is an error any intermediate directories previously created are removed; note this makes this method unsafe for concurrent use

writeTable

@Deprecated
public static void writeTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull File destFile)

Deprecated.

Use writeTable(Table, String) instead.

Write a table to a file. Data indexes to write are determined by those present on sourceTable.

Parameters:

sourceTable - source table

destFile - destination file; the file name should end in ".parquet" extension If the path includes non-existing directories they are created

writeTable

@Deprecated
public static void writeTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull File destFile,
 @NotNull
 @NotNull TableDefinition definition)

Deprecated.

Use writeTable(Table, String, ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Write a table to a file. Data indexes to write are determined by those present on sourceTable.

Parameters:

sourceTable - source table

destFile - destination file; its path must end in ".parquet". Any non existing directories in the path are created If there is an error any intermediate directories previously created are removed; note this makes this method unsafe for concurrent use

definition - table definition to use (instead of the one implied by the table itself)

writeTable

@Deprecated
public static void writeTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull File destFile,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Deprecated.

Use writeTable(Table, String, ParquetInstructions) instead.

Write a table to a file. Data indexes to write are determined by those present on sourceTable.

Parameters:

sourceTable - source table

destFile - destination file; its path must end in ".parquet". Any non existing directories in the path are created If there is an error any intermediate directories previously created are removed; note this makes this method unsafe for concurrent use

writeInstructions - instructions for customizations while writing

writeTable

public static void writeTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull String destination,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Write a table to a file. Data indexes to write are determined by those present on sourceTable.

Parameters:

sourceTable - source table

destination - destination path or URI; the file name should end in ".parquet" extension. If the path includes non-existing directories, they are created. If there is an error any intermediate directories previously created are removed; note this makes this method unsafe for concurrent use

writeInstructions - instructions for customizations while writing

writeTable

@Deprecated
public static void writeTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull String destination,
 @NotNull
 @NotNull TableDefinition definition,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Deprecated.

Use writeTable(Table, String, ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Write a table to a file. Data indexes to write are determined by those present on sourceTable.

Parameters:

sourceTable - source table

destination - destination path or URI; the file name should end in ".parquet" extension. If the path includes non-existing directories, they are created. If there is an error any intermediate directories previously created are removed; note this makes this method unsafe for concurrent use

definition - table definition to use (instead of the one implied by the table itself)

writeInstructions - instructions for customizations while writing

writeTable

@Deprecated
public static void writeTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull File destFile,
 @NotNull
 @NotNull TableDefinition definition,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Deprecated.

Use writeTable(Table, String, ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Write a table to a file. Data indexes to write are determined by those present on sourceTable.

Parameters:

sourceTable - source table

destFile - destination file; its path must end in ".parquet". Any non-existing directories in the path are created If there is an error any intermediate directories previously created are removed; note this makes this method unsafe for concurrent use

definition - table definition to use (instead of the one implied by the table itself)

writeInstructions - instructions for customizations while writing

legacyGroupingFileName

@VisibleForTesting
public static String legacyGroupingFileName(@NotNull
 @NotNull File tableDest,
 @NotNull
 @NotNull String columnName)

Legacy method for generating a grouping file name. We used to place grouping files right next to the original table destination.

Parameters:

tableDest - Destination path for the main table containing these grouping columns

columnName - Name of the grouping column

Returns:

The relative grouping file path. For example, for table with destination "table.parquet" and grouping column "GroupingColName", the method will return "table_GroupingColName_grouping.parquet"

writeKeyValuePartitionedTable

public static void writeKeyValuePartitionedTable(@NotNull
 @NotNull Table sourceTable,
 @NotNull
 @NotNull String destinationDir,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Write table to disk in parquet format with partitioning columns written as "key=value" format in a nested directory structure. To generate these individual partitions, this method will call partitionBy on all the partitioning columns of provided table. The generated parquet files will have names of the format provided by ParquetInstructions.baseNameForPartitionedParquetData(). By default, any indexing columns present on the source table will be written as sidecar tables. To write only a subset of the indexes or add additional indexes while writing, use ParquetInstructions.Builder.addIndexColumns(java.lang.String...).

Parameters:

sourceTable - The table to partition and write

destinationDir - The path to destination root directory to store partitioned data in nested format. Non-existing directories are created.

writeInstructions - Write instructions for customizations while writing

writeKeyValuePartitionedTable

public static void writeKeyValuePartitionedTable(@NotNull
 @NotNull PartitionedTable partitionedTable,
 @NotNull
 @NotNull String destinationDir,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Write a partitioned table to disk in parquet format with all the key columns as "key=value" format in a nested directory structure. To generate the partitioned table, users can call partitionBy on the required columns. The generated parquet files will have names of the format provided by ParquetInstructions.baseNameForPartitionedParquetData(). By default, this method does not write any indexes as sidecar tables to disk. To write such indexes, use ParquetInstructions.Builder.addIndexColumns(java.lang.String...).

Parameters:

partitionedTable - The partitioned table to write

destinationDir - The path to destination root directory to store partitioned data in nested format. Non-existing directories are created.

writeInstructions - Write instructions for customizations while writing

writeTables

@Deprecated
public static void writeTables(@NotNull
 @NotNull Table[] sources,
 @NotNull
 @NotNull TableDefinition definition,
 @NotNull
 @NotNull File[] destinations)

Deprecated.

Use writeTables(Table[], String[], ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Write out tables to disk. Data indexes to write are determined by those already present on the first source.

Parameters:

sources - source tables

definition - table definition

destinations - destinations

writeTables

public static void writeTables(@NotNull
 @NotNull Table[] sources,
 @NotNull
 @NotNull String[] destinations,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Write out tables to disk. Data indexes to write are determined by those already present on the first source or those provided through ParquetInstructions.Builder.addIndexColumns(java.lang.String...). The TableDefinition to use for writing must be provided as part of ParquetInstructions.

Parameters:

sources - The tables to write

destinations - The destination paths or URIs. Any non-existing directories in the paths provided are created. If there is an error, any intermediate directories previously created are removed; note this makes this method unsafe for concurrent use.

writeInstructions - Write instructions for customizations while writing

writeTables

@Deprecated
public static void writeTables(@NotNull
 @NotNull Table[] sources,
 @NotNull
 @NotNull TableDefinition definition,
 @NotNull
 @NotNull File[] destinations,
 @NotNull
 @NotNull ParquetInstructions writeInstructions)

Deprecated.

Use writeTables(Table[], String[], ParquetInstructions) instead with TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Write out tables to disk. Data indexes to write are determined by those already present on the first source.

Parameters:

sources - source tables

definition - table definition

destinations - destinations

writeInstructions - instructions for customizations while writing

deleteTable

@VisibleForTesting
public static void deleteTable(File path)

Deletes a table on disk.

Parameters:

path - path to delete

readSingleFileTable

@Deprecated
public static Table readSingleFileTable(@NotNull
 @NotNull ParquetTableLocationKey tableLocationKey,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Reads in a table from a single parquet file using the table definition provided through the ParquetInstructions.

Parameters:

tableLocationKey - The location keys to include

readInstructions - Instructions for customizations while reading

Returns:

The table

readSingleFileTable

@Deprecated
public static Table readSingleFileTable(@NotNull
 @NotNull ParquetTableLocationKey tableLocationKey,
 @NotNull
 @NotNull ParquetInstructions readInstructions,
 @NotNull
 @NotNull TableDefinition tableDefinition)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition)

Reads in a table from a single parquet file using the provided table definition.

Parameters:

tableLocationKey - The location keys to include

readInstructions - Instructions for customizations while reading

tableDefinition - The table's definition

Returns:

The table

readTable

public static Table readTable(@NotNull
 @NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Reads in a table from files discovered with locationKeyFinder using a definition either provided using ParquetInstructions or built from the highest (by location key order) location found, which must have non-null partition values for all partition keys.

Callers may prefer the simpler methods readTable(String, ParquetInstructions) with layout provided using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Parameters:

locationKeyFinder - The source of location keys to include

readInstructions - Instructions for customizations while reading

Returns:

The table

readPartitionedTable

@Deprecated
public static Table readPartitionedTable(@NotNull
 @NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

use readTable(TableLocationKeyFinder, ParquetInstructions) instead

Reads in a table from files discovered with locationKeyFinder using a definition either provided using ParquetInstructions or built from the highest (by location key order) location found, which must have non-null partition values for all partition keys.

Callers may prefer the simpler methods readTable(String, ParquetInstructions) with layout provided using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Parameters:

locationKeyFinder - The source of location keys to include

readInstructions - Instructions for customizations while reading

Returns:

The table

readPartitionedTable

@Deprecated
public static Table readPartitionedTable(@NotNull
 @NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder,
 @NotNull
 @NotNull ParquetInstructions readInstructions,
 @NotNull
 @NotNull TableDefinition tableDefinition)

Deprecated.

use readTable(TableLocationKeyFinder, ParquetInstructions) instead with the table definition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Reads in a table from files discovered with locationKeyFinder using the provided table definition.

Callers may prefer the simpler methods readTable(String, ParquetInstructions) with layout provided using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Parameters:

locationKeyFinder - The source of location keys to include

readInstructions - Instructions for customizations while reading

tableDefinition - The table's definition

Returns:

The table

readPartitionedTableInferSchema

@Deprecated
public static Table readPartitionedTableInferSchema(@NotNull
 @NotNull TableLocationKeyFinder<ParquetTableLocationKey> locationKeyFinder,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

use readTable(TableLocationKeyFinder, ParquetInstructions)

Reads in a table from files discovered with locationKeyFinder using a definition built from the highest (by location key order) location found, which must have non-null partition values for all partition keys.

Callers may prefer the simpler methods readTable(String, ParquetInstructions) with layout provided using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Parameters:

locationKeyFinder - The source of location keys to include

readInstructions - Instructions for customizations while reading

Returns:

The table

readPartitionedTableWithMetadata

@Deprecated
public static Table readPartitionedTableWithMetadata(@NotNull
 @NotNull File directory,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.METADATA_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Reads in a table using metadata files found in the supplied directory.

Parameters:

directory - the path for the root directory to search for .parquet files

readInstructions - Instructions for customizations while reading

Returns:

The table

readPartitionedTableWithMetadata

@Deprecated
public static Table readPartitionedTableWithMetadata(@NotNull
 @NotNull String directory,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.METADATA_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Reads in a table using metadata files found in the supplied directory.

Parameters:

directory - the path or URI for the root directory to search for .parquet files

readInstructions - Instructions for customizations while reading

Returns:

The table

readKeyValuePartitionedTable

@Deprecated
public static Table readKeyValuePartitionedTable(@NotNull
 @NotNull File directory,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.KV_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Creates a partitioned table via the key-value partitioned parquet files from the root directory, inferring the table definition from those files.

Callers wishing to be more explicit and skip the inference step may prefer to call readKeyValuePartitionedTable(File, ParquetInstructions, TableDefinition).

Parameters:

directory - the root directory to search for .parquet files

readInstructions - the instructions for customizations while reading

Returns:

the table

readKeyValuePartitionedTable

@Deprecated
public static Table readKeyValuePartitionedTable(@NotNull
 @NotNull File directory,
 @NotNull
 @NotNull ParquetInstructions readInstructions,
 @NotNull
 @NotNull TableDefinition tableDefinition)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.KV_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Creates a partitioned table via the key-value partitioned parquet files from the root directory using the provided tableDefinition.

Parameters:

directory - the root directory to search for .parquet files

readInstructions - the instructions for customizations while reading

tableDefinition - the table definition

Returns:

the table

readFlatPartitionedTable

@Deprecated
public static Table readFlatPartitionedTable(@NotNull
 @NotNull File directory,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.FLAT_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Creates a partitioned table via the flat parquet files from the root directory, inferring the table definition from those files.

Callers wishing to be more explicit and skip the inference step may prefer to call readFlatPartitionedTable(File, ParquetInstructions, TableDefinition).

Parameters:

directory - the directory to search for .parquet files

readInstructions - the instructions for customizations while reading

Returns:

the table

readFlatPartitionedTable

@Deprecated
public static Table readFlatPartitionedTable(@NotNull
 @NotNull File directory,
 @NotNull
 @NotNull ParquetInstructions readInstructions,
 @NotNull
 @NotNull TableDefinition tableDefinition)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.FLAT_PARTITIONED using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Creates a partitioned table via the flat parquet files from the root directory using the provided tableDefinition.

Parameters:

directory - the directory to search for .parquet files

readInstructions - the instructions for customizations while reading

tableDefinition - the table definition

Returns:

the table

readSingleFileTable

@Deprecated
public static Table readSingleFileTable(@NotNull
 @NotNull File file,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Creates a single table via the parquet file using the table definition derived from that file.

Callers wishing to be more explicit (for example, to skip some columns) may prefer to call readSingleFileTable(File, ParquetInstructions, TableDefinition).

Parameters:

file - the parquet file

readInstructions - the instructions for customizations while reading

Returns:

the table

readSingleFileTable

@Deprecated
public static Table readSingleFileTable(@NotNull
 @NotNull String source,
 @NotNull
 @NotNull ParquetInstructions readInstructions)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout).

Creates a single table via the parquet source using the provided tableDefinition. The source provided can be a local file path or a URI to be resolved via the provided SeekableChannelsProviderPlugin

Callers wishing to be more explicit (for example, to skip some columns) may prefer to call readSingleFileTable(String, ParquetInstructions, TableDefinition).

Parameters:

source - the path or URI for the parquet file

readInstructions - the instructions for customizations while reading

Returns:

the table

readSingleFileTable

@Deprecated
public static Table readSingleFileTable(@NotNull
 @NotNull File file,
 @NotNull
 @NotNull ParquetInstructions readInstructions,
 @NotNull
 @NotNull TableDefinition tableDefinition)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Creates a single table via the parquet file using the provided tableDefinition.

Parameters:

file - the parquet file

readInstructions - the instructions for customizations while reading

tableDefinition - the table definition

Returns:

the table

readSingleFileTable

@Deprecated
public static Table readSingleFileTable(@NotNull
 @NotNull String source,
 @NotNull
 @NotNull ParquetInstructions readInstructions,
 @NotNull
 @NotNull TableDefinition tableDefinition)

Deprecated.

Use readTable(String, ParquetInstructions) instead with layout provided as ParquetInstructions.ParquetFileLayout.SINGLE_FILE using ParquetInstructions.Builder.setFileLayout(io.deephaven.parquet.table.ParquetInstructions.ParquetFileLayout) and TableDefinition provided through ParquetInstructions.Builder.setTableDefinition(io.deephaven.engine.table.TableDefinition).

Creates a single table via the parquet source using the provided tableDefinition. The source provided can be a local file path or a URI to be resolved via the provided SeekableChannelsProviderPlugin

Parameters:

source - the path or URI for the parquet file

readInstructions - the instructions for customizations while reading

tableDefinition - the table definition

Returns:

the table

readParquetSchemaAndTable

@VisibleForTesting
public static Table readParquetSchemaAndTable(@NotNull
 @NotNull File source,
 @NotNull
 @NotNull ParquetInstructions readInstructionsIn,
 @Nullable
 @Nullable org.apache.commons.lang3.mutable.MutableObject<ParquetInstructions> mutableInstructionsOut)

setDefaultCompressionCodecName

@Deprecated
public static void setDefaultCompressionCodecName(String compressionCodecName)

Deprecated.

Do not use this method, instead pass the above codecs as arguments to writeTable(Table, String, ParquetInstructions) method