Package io.deephaven.extensions.barrage.chunk

Interface ChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>>

Type Parameters:
SOURCE_CHUNK_TYPE - The type of chunk of source data, extending Chunk with Values.
All Known Implementing Classes:
BaseChunkWriter, BigDecimalChunkWriter, BooleanChunkWriter, ByteChunkWriter, CharChunkWriter, DictionaryChunkWriter, DoubleChunkWriter, FixedWidthChunkWriter, FixedWidthObjectChunkWriter, FloatChunkWriter, IntChunkWriter, ListChunkWriter, LongChunkWriter, MapChunkWriter, NullChunkWriter, RunEndEncodedChunkWriter, ShortChunkWriter, UnionChunkWriter, VarBinaryChunkWriter
public interface ChunkWriter<SOURCE_CHUNK_TYPE extends Chunk<Values>>
The ChunkWriter interface provides a mechanism for writing chunks of data into a structured format suitable for transmission in Apache Arrow's columnar format. It enables efficient handling of chunked data, including support for various data types and logical structures. This interface is part of the Deephaven Barrage extensions for efficient data streaming and processing.

  • Method Details

    • makeContext

      ChunkWriter.Context makeContext(@NotNull SOURCE_CHUNK_TYPE chunk, long rowOffset)
      Create a context for the given chunk.
      Parameters:
      chunk - the chunk of data to be written
      rowOffset - the offset into the logical message potentially spread over multiple chunks
      Returns:
      a context for the given chunk

    • getInputStream

      ChunkWriter.DrainableColumn getInputStream(@NotNull @NotNull ChunkWriter.Context context, @Nullable @Nullable RowSet subset, @NotNull @NotNull BarrageOptions options) throws IOException
      Get an input stream optionally position-space filtered using the provided RowSet.
      Parameters:
      context - the chunk writer context holding the data to be drained to the client
      subset - if provided, is a position-space filter of source data
      options - options for writing to the stream
      Returns:
      a single-use DrainableColumn ready to be drained via grpc
      Throws:
      IOException

    • getInputStream

      default ChunkWriter.DrainableColumn getInputStream(@NotNull @NotNull ChunkWriter.Context context, @Nullable @Nullable RowSet subset, @NotNull @NotNull BarrageOptions options, @Nullable @Nullable DictionaryWriterRegistry dictionaryRegistry) throws IOException
      Get an input stream, threading a DictionaryWriterRegistry down to any dictionary-encoded writers nested within this writer (e.g. the values child of a run-end-encoded column). Writers that contain no dictionary encoding ignore the registry and behave identically to getInputStream(Context, RowSet, BarrageOptions).

      A DictionaryChunkWriter uses the registry to obtain (and register) its per-id DictionaryWriterState, so that the enclosing message writer emits the corresponding DictionaryBatch. Composite writers forward the registry to their children.

      Parameters:
      context - the chunk writer context holding the data to be drained to the client
      subset - if provided, is a position-space filter of source data
      options - options for writing to the stream
      dictionaryRegistry - the per-stream dictionary registry; may be null when the caller does not support dictionary encoding (a nested dictionary-encoded writer will then throw)
      Returns:
      a single-use DrainableColumn ready to be drained via grpc
      Throws:
      IOException

    • getEmptyInputStream

      ChunkWriter.DrainableColumn getEmptyInputStream(@NotNull @NotNull BarrageOptions options) throws IOException
      Get an input stream representing the empty wire payload for this writer.
      Parameters:
      options - options for writing to the stream
      Returns:
      a single-use DrainableColumn ready to be drained via grpc
      Throws:
      IOException

    • getEmptyInputStream

      default ChunkWriter.DrainableColumn getEmptyInputStream(@NotNull @NotNull BarrageOptions options, @Nullable @Nullable DictionaryWriterRegistry dictionaryRegistry) throws IOException
      Get an input stream representing the empty wire payload for this writer, threading a DictionaryWriterRegistry down to any dictionary-encoded writer nested within this writer, mirroring getInputStream(Context, RowSet, BarrageOptions, DictionaryWriterRegistry). This matters even for an empty payload: a dictionary-encoded column whose very first batch carries no rows (e.g. a freshly-created ticking table's initial, still-empty snapshot) must still register its id and emit an initial isDelta=false DictionaryBatch before any RecordBatch references it. Writers that contain no dictionary encoding ignore the registry and behave identically to getEmptyInputStream(BarrageOptions).
      Parameters:
      options - options for writing to the stream
      dictionaryRegistry - the per-stream dictionary registry; may be null when the caller does not support dictionary encoding (a nested dictionary-encoded writer will then throw)
      Returns:
      a single-use DrainableColumn ready to be drained via grpc
      Throws:
      IOException

    • isFieldNullable

      boolean isFieldNullable()
      Returns:
      whether the wire format for this writer might include a validity buffer