Package io.deephaven.parquet.table.transfer

Class DictEncodedStringArrayAndVectorTransfer<T>

java.lang.Object
io.deephaven.parquet.table.transfer.ArrayAndVectorTransfer<COLUMN_TYPE,ENCODED_COLUMN_TYPE,BUFFER_TYPE>
io.deephaven.parquet.table.transfer.DictEncodedStringArrayAndVectorTransfer<T>
All Implemented Interfaces:
TransferObject<IntBuffer>, SafeCloseable, AutoCloseable
public abstract class DictEncodedStringArrayAndVectorTransfer<T> extends ArrayAndVectorTransfer<COLUMN_TYPE,ENCODED_COLUMN_TYPE,BUFFER_TYPE>
Base class for reading dictionary-encoded string arrays and vectors. This class updates the StringDictionary with all the strings it encounters and generates an integer array of dictionary position values. This class extends PrimitiveArrayAndVectorTransfer to manage the dictionary positions similar to an integer array column.

  • Method Details

    • transferOnePageToBuffer

      public final int transferOnePageToBuffer()
      Description copied from interface: TransferObject
      Transfer one page size worth of fetched data into an internal buffer, which can then be accessed using TransferObject.getBuffer(). The target page size is passed in the constructor. For dictionary encoded string transfers, this method also updates the dictionary with the strings encountered.
      Specified by:
      transferOnePageToBuffer in interface TransferObject<T>
      Returns:
      The number of fetched data entries copied into the buffer. This can be different from the total number of entries fetched in case of variable-width types (e.g. strings) when used with additional page size limits while copying.

    • pageHasNull

      public final boolean pageHasNull()
      Description copied from interface: TransferObject
      Returns whether we encountered any null value while transferring page data to buffer. This method is only used for dictionary encoded string transfer objects. This method should be called after TransferObject.transferOnePageToBuffer() and the state resets everytime we call TransferObject.transferOnePageToBuffer().

    • getBuffer

      public final IntBuffer getBuffer()
      Description copied from interface: TransferObject
      Get the buffer suitable for writing to a Parquet file
      Specified by:
      getBuffer in interface TransferObject<COLUMN_TYPE>
      Returns:
      the buffer

    • hasMoreDataToBuffer

      public final boolean hasMoreDataToBuffer()
      Description copied from interface: TransferObject
      Check if there is any more data which can be copied into buffer
      Specified by:
      hasMoreDataToBuffer in interface TransferObject<COLUMN_TYPE>

    • close

      public final void close()
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface SafeCloseable