Package io.deephaven.extensions.barrage.chunk

Class BaseChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>>

java.lang.Object
io.deephaven.extensions.barrage.chunk.BaseChunkWriter<SOURCE_CHUNK_TYPE>
All Implemented Interfaces:
ChunkWriter<SOURCE_CHUNK_TYPE>
Direct Known Subclasses:
BooleanChunkWriter, ByteChunkWriter, CharChunkWriter, DictionaryChunkWriter, DoubleChunkWriter, FixedWidthChunkWriter, FloatChunkWriter, IntChunkWriter, ListChunkWriter, LongChunkWriter, MapChunkWriter, NullChunkWriter, RunEndEncodedChunkWriter, ShortChunkWriter, UnionChunkWriter, VarBinaryChunkWriter
public abstract class BaseChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>> extends Object implements ChunkWriter<SOURCE_CHUNK_TYPE>

  • Field Details

    • PADDING_BUFFER

      public static final byte[] PADDING_BUFFER

    • REMAINDER_MOD_8_MASK

      public static final int REMAINDER_MOD_8_MASK
      See Also:

    • BULK_WRITE_BUFFER_BYTES

      protected static final int BULK_WRITE_BUFFER_BYTES
      Upper bound (in bytes) on a single bulk payload write shared by the fixed-width primitive writers. Wide columns are encoded into little-endian bytes and flushed in windows of this size rather than writing one value (and its individual bytes) at a time. Configure once here (rather than per type) via BaseChunkWriter.bulkWriteBufferBytes.

    • elementSize

      protected final int elementSize
      the size of each element in bytes if fixed

    • dhNullable

      protected final boolean dhNullable
      whether we can use the wire value as a deephaven null for clients that support dh nulls

    • fieldNullable

      protected final boolean fieldNullable
      whether the field is nullable

  • Method Details

    • getEmptyInputStream

      public final ChunkWriter.DrainableColumn getEmptyInputStream(@NotNull @NotNull BarrageOptions options) throws IOException
      Description copied from interface: ChunkWriter
      Get an input stream representing the empty wire payload for this writer.
      Specified by:
      getEmptyInputStream in interface ChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>>
      Parameters:
      options - options for writing to the stream
      Returns:
      a single-use DrainableColumn ready to be drained via grpc
      Throws:
      IOException

    • getEmptyInputStream

      public ChunkWriter.DrainableColumn getEmptyInputStream(@NotNull @NotNull BarrageOptions options, @Nullable @Nullable DictionaryWriterRegistry dictionaryRegistry) throws IOException
      Description copied from interface: ChunkWriter
      Get an input stream representing the empty wire payload for this writer, threading a DictionaryWriterRegistry down to any dictionary-encoded writer nested within this writer, mirroring ChunkWriter.getInputStream(Context, RowSet, BarrageOptions, DictionaryWriterRegistry). This matters even for an empty payload: a dictionary-encoded column whose very first batch carries no rows (e.g. a freshly-created ticking table's initial, still-empty snapshot) must still register its id and emit an initial isDelta=false DictionaryBatch before any RecordBatch references it. Writers that contain no dictionary encoding ignore the registry and behave identically to ChunkWriter.getEmptyInputStream(BarrageOptions).
      Specified by:
      getEmptyInputStream in interface ChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>>
      Parameters:
      options - options for writing to the stream
      dictionaryRegistry - the per-stream dictionary registry; may be null when the caller does not support dictionary encoding (a nested dictionary-encoded writer will then throw)
      Returns:
      a single-use DrainableColumn ready to be drained via grpc
      Throws:
      IOException

    • makeContext

      public ChunkWriter.Context makeContext(@NotNull SOURCE_CHUNK_TYPE chunk, long rowOffset)
      Description copied from interface: ChunkWriter
      Create a context for the given chunk.
      Specified by:
      makeContext in interface ChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>>
      Parameters:
      chunk - the chunk of data to be written
      rowOffset - the offset into the logical message potentially spread over multiple chunks
      Returns:
      a context for the given chunk

    • isFieldNullable

      public boolean isFieldNullable()
      Specified by:
      isFieldNullable in interface ChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>>
      Returns:
      whether the wire format for this writer might include a validity buffer

    • computeValidity

      protected abstract void computeValidity(@NotNull @NotNull ChunkWriter.Context context, @NotNull @NotNull RowSequence subset, @NotNull @NotNull BaseChunkWriter.ValidityBuffer validity)
      Report the nullness of each row of the subset, in order, to validity.

      This is invoked at most once per BaseChunkWriter.BaseChunkInputStream: the null count carried by the field node and the bytes of the validity buffer are both derived from this single traversal of the row data.

      Parameters:
      context - the context for the chunk
      subset - the subset of rows to consider
      validity - the validity buffer to populate

    • getValidityMapSerializationSizeFor

      protected static int getValidityMapSerializationSizeFor(int numElements)
      Returns expected size of validity map in bytes.
      Parameters:
      numElements - the number of rows
      Returns:
      number of bytes to represent the validity buffer for numElements

    • getNumLongsForBitPackOfSize

      protected static int getNumLongsForBitPackOfSize(int numElements)
      Returns the number of longs needed to represent a single bit per element.
      Parameters:
      numElements - the number of rows
      Returns:
      number of longs needed to represent numElements bits rounded up to the nearest long