Package io.deephaven.extensions.barrage.chunk

Interface DictionaryWriterState

All Known Implementing Classes:
LocalDictionaryWriterState, SharedDictionaryWriterState
public interface DictionaryWriterState
Tracks the cumulative dictionary for one Arrow dictionary id within a barrage stream. Shared by all DictionaryChunkWriter instances that reference the same id.

fillIndexChunk(io.deephaven.chunk.Chunk<io.deephaven.chunk.attributes.Values>, io.deephaven.engine.rowset.RowSet, io.deephaven.extensions.barrage.BarrageOptions, io.deephaven.chunk.WritableIntChunk<io.deephaven.chunk.attributes.Values>) is called once per batch (non-null rows only) while building a batch. After the batch's DictionaryBatch has been emitted, call resetDelta() to advance the delta boundary.

Two concrete implementations exist:

  • LocalDictionaryWriterState — for viewport subscriptions and snapshots. resetDelta() clears the delta list entirely; only newly-seen values since the last reset are tracked.
  • SharedDictionaryWriterState — for full subscriptions and growing subscriptions. The full cumulative value list is retained so that a new subscriber joining mid-stream can receive all current values as an initial isDelta=false batch.

Thread-safety: not thread-safe; single-threaded barrage stream serialization is assumed.

  • Method Summary

    Modifier and Type
    Method
    Description
    @NotNull WritableChunk<Values>
    buildDeltaChunk()
    Builds and returns a typed chunk containing the current delta values (values added since the last resetDelta() call, or all values if this is the first batch for this subscriber).
    void
    fillIndexChunk(@NotNull Chunk<Values> source, @Nullable RowSet subset, @NotNull BarrageOptions options, @NotNull WritableIntChunk<Values> out)
    Fills out with one dictionary index per logical row in source/subset.
    long
    getDictId()
     
    boolean
    hasDelta()
    Returns true if a DictionaryBatch message needs to be emitted before the current RecordBatch — either because this is the first batch for this subscriber (isDelta=false) or because new values have been added since the last reset.
    boolean
    needsFullBatch()
     
    void
    reset()
    Resets the dictionary to an empty state, as if no values had ever been seen.
    void
    resetDelta()
    Advances the delta boundary after a DictionaryBatch has been successfully emitted.
    int
    totalSize()
    Current number of distinct values in the dictionary (resets to zero after reset()).

  • Method Details

    • getDictId

      long getDictId()

    • fillIndexChunk

      void fillIndexChunk(@NotNull @NotNull Chunk<Values> source, @Nullable @Nullable RowSet subset, @NotNull @NotNull BarrageOptions options, @NotNull @NotNull WritableIntChunk<Values> out)
      Fills out with one dictionary index per logical row in source/subset. Null rows (in non-deephaven-nulls mode) produce QueryConstants.NULL_INT; non-null rows produce a non-negative dictionary index, registering new values as needed.
      Parameters:
      source - the source chunk containing column values
      subset - row positions within source to include; null means all rows
      options - barrage serialization options (e.g. useDeephavenNulls)
      out - pre-sized output chunk to fill with dictionary indices

    • hasDelta

      boolean hasDelta()
      Returns true if a DictionaryBatch message needs to be emitted before the current RecordBatch — either because this is the first batch for this subscriber (isDelta=false) or because new values have been added since the last reset.

    • needsFullBatch

      boolean needsFullBatch()

    • buildDeltaChunk

      @NotNull @NotNull WritableChunk<Values> buildDeltaChunk()
      Builds and returns a typed chunk containing the current delta values (values added since the last resetDelta() call, or all values if this is the first batch for this subscriber). The returned chunk is owned by the caller and must be closed when no longer needed.

    • resetDelta

      void resetDelta()
      Advances the delta boundary after a DictionaryBatch has been successfully emitted. Unlike reset(), this does not discard the accumulated value-to-index mapping — it only moves the boundary so that already-sent values are excluded from future delta batches. The client's cached dictionary remains valid after this call.

    • totalSize

      int totalSize()
      Current number of distinct values in the dictionary (resets to zero after reset()).

    • reset

      void reset()
      Resets the dictionary to an empty state, as if no values had ever been seen. The next DictionaryBatch emitted will be isDelta=false with only the values encountered in the next batch. Call this when the cumulative dictionary size exceeds the live row count and compaction is needed.