Package io.deephaven.extensions.barrage.chunk

Interface DictionaryWriterState

All Known Implementing Classes:
LocalDictionaryWriterState, SharedDictionaryWriterState
public interface DictionaryWriterState
Tracks the cumulative dictionary for one Arrow dictionary id within a barrage stream. Shared by all DictionaryChunkWriter instances that reference the same id.

fillIndexChunk(io.deephaven.chunk.Chunk<io.deephaven.chunk.attributes.Values>, io.deephaven.engine.rowset.RowSet, io.deephaven.extensions.barrage.BarrageOptions, io.deephaven.chunk.WritableIntChunk<io.deephaven.chunk.attributes.Values>) is called once per batch (non-null rows only) while building a batch. After the batch's DictionaryBatch has been emitted, call resetDelta() to advance the delta boundary.

Two concrete implementations exist:

  • LocalDictionaryWriterState — for viewport subscriptions and snapshots. resetDelta() clears the delta list entirely; only newly-seen values since the last reset are tracked.
  • SharedDictionaryWriterState — for full subscriptions and growing subscriptions. The full cumulative value list is retained so that a new subscriber joining mid-stream can receive all current values as an initial isDelta=false batch.

Thread-safety: an instance serves a single subscriber, whose writes use the instance from one thread at a time, so the instance does not need to be thread-safe. The SharedDictionaryWriterState instances of different subscribers share a SharedWriterDictionary, which is thread-safe.

  • Method Summary

    Modifier and Type
    Method
    Description
    @NotNull WritableChunk<Values>
    buildDeltaChunk()
    Builds and returns a typed chunk containing the current delta values (values added since the last resetDelta() call, or all values if this is the first batch for this subscriber).
    void
    fillIndexChunk(@NotNull Chunk<Values> source, @Nullable RowSet subset, @NotNull BarrageOptions options, @NotNull WritableIntChunk<Values> out)
    Fills out with one dictionary index per logical row in source/subset.
    long
    getDictId()
     
    boolean
    hasDelta()
    Returns true if a DictionaryBatch message needs to be emitted before the current RecordBatch — either because this is the first batch for this subscriber (isDelta=false) or because new values have been added since the last reset.
    boolean
    needsFullBatch()
     
    void
    reset()
    Resets the dictionary to an empty state, as if no values had ever been seen.
    void
    resetDelta()
    Accepts the batch just written.
    int
    totalSize()
    Current number of distinct values in the dictionary (resets to zero after reset()).

  • Method Details

    • getDictId

      long getDictId()

    • fillIndexChunk

      void fillIndexChunk(@NotNull @NotNull Chunk<Values> source, @Nullable @Nullable RowSet subset, @NotNull @NotNull BarrageOptions options, @NotNull @NotNull WritableIntChunk<Values> out)
      Fills out with one dictionary index per logical row in source/subset. Null rows (in non-deephaven-nulls mode) produce QueryConstants.NULL_INT; non-null rows produce a non-negative dictionary index, registering new values as needed.
      Parameters:
      source - the source chunk containing column values
      subset - row positions within source to include; null means all rows
      options - barrage serialization options (e.g. useDeephavenNulls)
      out - pre-sized output chunk to fill with dictionary indices

    • hasDelta

      boolean hasDelta()
      Returns true if a DictionaryBatch message needs to be emitted before the current RecordBatch — either because this is the first batch for this subscriber (isDelta=false) or because new values have been added since the last reset. A subscriber is owed values that others added to a shared dictionary only once a batch of its own references the dictionary; see SharedDictionaryWriterState.

    • needsFullBatch

      boolean needsFullBatch()

    • buildDeltaChunk

      @NotNull @NotNull WritableChunk<Values> buildDeltaChunk()
      Builds and returns a typed chunk containing the current delta values (values added since the last resetDelta() call, or all values if this is the first batch for this subscriber). The returned chunk is owned by the caller and must be closed when no longer needed.

    • resetDelta

      void resetDelta()
      Accepts the batch just written. Advances the delta boundary past the values that the last buildDeltaChunk() covered, once the DictionaryBatch built from them has been emitted, so that they are excluded from future delta batches. This is called after every accepted batch, whether or not a DictionaryBatch was built for this state; without a preceding buildDeltaChunk() call, the shared implementation's boundary does not move. Unlike reset(), this does not discard the accumulated value-to-index mapping, so the client's cached dictionary remains valid after this call.

    • totalSize

      int totalSize()
      Current number of distinct values in the dictionary (resets to zero after reset()).

    • reset

      void reset()
      Resets the dictionary to an empty state, as if no values had ever been seen. The next DictionaryBatch emitted will be isDelta=false with only the values encountered in the next batch. Call this when the cumulative dictionary size exceeds the live row count and compaction is needed.