Package io.deephaven.parquet.compress

Interface CompressorAdapter

All Superinterfaces:

AutoCloseable, SafeCloseable

public interface CompressorAdapter
extends SafeCloseable

An Intermediate adapter interface between Deephaven column writing and parquet compression.

Field Summary

Fields

Modifier and Type	Field	Description
static final CompressorAdapter	PASSTHRU	An CompressorAdapter instance that reads and writes uncompressed data directly.

Method Summary

Modifier and Type	Method	Description
OutputStream	compress(OutputStream os)	Creates a new output stream that will take uncompressed writes, and flush data to the provided stream as compressed data.
org.apache.parquet.bytes.BytesInput	decompress(InputStream inputStream, int compressedSize, int uncompressedSize, Function<Supplier<SafeCloseable>,SafeCloseable> decompressorCache)	Returns an in-memory instance of BytesInput containing the fully decompressed results of the input stream.
org.apache.parquet.hadoop.metadata.CompressionCodecName	getCodecName()
void	reset()	Reset the internal state of this CompressorAdapter so more rows can be read or written.

Methods inherited from interface io.deephaven.util.SafeCloseable

close

Field Details

PASSTHRU

static final CompressorAdapter PASSTHRU

An CompressorAdapter instance that reads and writes uncompressed data directly.

Method Details

compress

OutputStream compress(OutputStream os)
               throws IOException

Creates a new output stream that will take uncompressed writes, and flush data to the provided stream as compressed data.

Note that this method is not thread safe.

Parameters:

os - the output stream to write compressed contents to

Returns:

an output stream that can accept writes

Throws:

IOException - thrown if an error occurs writing data

decompress

org.apache.parquet.bytes.BytesInput decompress(InputStream inputStream,
 int compressedSize,
 int uncompressedSize,
 Function<Supplier<SafeCloseable>,SafeCloseable> decompressorCache)
                                        throws IOException

Returns an in-memory instance of BytesInput containing the fully decompressed results of the input stream. The provided DecompressorHolder is used for decompressing if compatible with the compression codec. Otherwise, a new decompressor is created and set in the DecompressorHolder.

Note that this method is thread safe, assuming the cached decompressor instances are not shared across threads.

Parameters:

inputStream - an input stream containing compressed data

compressedSize - the number of bytes in the compressed data

uncompressedSize - the number of bytes that should be present when decompressed

decompressorCache - Used to cache Decompressor instances for reuse

Returns:

the decompressed bytes, copied into memory

Throws:

IOException - thrown if an error occurs reading data.

getCodecName

org.apache.parquet.hadoop.metadata.CompressionCodecName getCodecName()

Returns:

the CompressionCodecName enum value that represents this compressor.

reset

void reset()

Reset the internal state of this CompressorAdapter so more rows can be read or written.

This method can be called after compress(java.io.OutputStream) to reset the internal state of the compressor, and is not required before compress(java.io.OutputStream), or before and after decompress(java.io.InputStream, int, int, java.util.function.Function<java.util.function.Supplier<io.deephaven.util.SafeCloseable>, io.deephaven.util.SafeCloseable>) because those methods internally manage their own state.