Package io.deephaven.parquet.base

Interface BulkWriter<BUFFER_TYPE>

All Known Implementing Classes:
AbstractBulkValuesWriter, PlainBinaryChunkedWriter, PlainBooleanChunkedWriter, PlainDoubleChunkedWriter, PlainFloatChunkedWriter, PlainIntChunkedWriter, PlainLongChunkedWriter, RleIntChunkedWriter
public interface BulkWriter<BUFFER_TYPE>
Provides the option to write values of specific type in bulk The concrete type of the bulkValue object depends on the specific implementation

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static class 
    BulkWriter.WriteResult
     

  • Method Summary

    Modifier and Type
    Method
    Description
    ByteBuffer
    getByteBufferView()
    Get a view of this writer ByteBuffer.
    void
    reset()
    Clear all internal state.
    void
    writeBulk(BUFFER_TYPE bulkValues, int rowCount, @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics)
    Write a buffer's worth of values to the underlying page.
    @NotNull BulkWriter.WriteResult
    writeBulkFilterNulls(BUFFER_TYPE bulkValues, @NotNull org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder dlEncoder, int rowCount, @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics)
    Write a buffer's worth of values to the underlying page.
    int
    writeBulkVector(BUFFER_TYPE bulkValues, @NotNull IntBuffer vectorSizes, @NotNull org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder rlEncoder, @NotNull org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder dlEncoder, int nonNullValueCount, @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics)
    Write a buffer's worth of packed vector values to the underlying page.
    @NotNull BulkWriter.WriteResult
    writeBulkVectorFilterNulls(BUFFER_TYPE bulkValues, int rowCount, @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics)
    Write a buffer's worth of packed vector values to the underlying page, skipping null values.

  • Method Details

    • writeBulk

      void writeBulk(@NotNull BUFFER_TYPE bulkValues, int rowCount, @NotNull @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics)
      Write a buffer's worth of values to the underlying page.
      Parameters:
      bulkValues - the buffer of values
      rowCount - the total number of rows to write.

    • writeBulkFilterNulls

      @NotNull @NotNull BulkWriter.WriteResult writeBulkFilterNulls(@NotNull BUFFER_TYPE bulkValues, @NotNull @NotNull org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder dlEncoder, int rowCount, @NotNull @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics) throws IOException
      Write a buffer's worth of values to the underlying page. This method will find, without writing, null values and record their offsets in an IntBuffer in the result. The appropriate definition level will be set for null values.
      Parameters:
      bulkValues - the values to write
      dlEncoder - the encoder for definition levels
      rowCount - the number of rows being written
      statistics - the Statistics object to modify.
      Returns:
      a BulkWriter.WriteResult containing the statistics of the result.
      Throws:
      IOException - if there was an error during write.

    • writeBulkVector

      int writeBulkVector(@NotNull BUFFER_TYPE bulkValues, @NotNull @NotNull IntBuffer vectorSizes, @NotNull @NotNull org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder rlEncoder, @NotNull @NotNull org.apache.parquet.column.values.rle.RunLengthBitPackingHybridEncoder dlEncoder, int nonNullValueCount, @NotNull @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics) throws IOException
      Write a buffer's worth of packed vector values to the underlying page. This method will set the proper definition level and repetition values in the encoders for null values.
      Parameters:
      bulkValues - the packed array values
      vectorSizes - a buffer where each element contains the number of elements in each packed vector.
      rlEncoder - the repetition level encoder
      dlEncoder - the definition level encoder.
      nonNullValueCount - the total count of non-null values
      Returns:
      the number of values actually written
      Throws:
      IOException - if writing failed.

    • writeBulkVectorFilterNulls

      @NotNull @NotNull BulkWriter.WriteResult writeBulkVectorFilterNulls(@NotNull BUFFER_TYPE bulkValues, int rowCount, @NotNull @NotNull org.apache.parquet.column.statistics.Statistics<?> statistics)
      Write a buffer's worth of packed vector values to the underlying page, skipping null values. This method will find null values and record their offsets in an IntBuffer in the result.
      Parameters:
      bulkValues - the packed vector values to write
      rowCount - the number of rows being written.
      Returns:
      a BulkWriter.WriteResult containing the statistics of the result.

    • reset

      void reset()
      Clear all internal state.

    • getByteBufferView

      ByteBuffer getByteBufferView() throws IOException
      Get a view of this writer ByteBuffer.
      Returns:
      a ByteBuffer containing the written data.
      Throws:
      IOException - if there is an exception reading the data.