Package io.deephaven.kafka.ingest

Class GenericRecordConsumerRecordToTableWriterAdapter.Builder

java.lang.Object
io.deephaven.kafka.ingest.GenericRecordConsumerRecordToTableWriterAdapter.Builder
Enclosing class:
GenericRecordConsumerRecordToTableWriterAdapter
public static class GenericRecordConsumerRecordToTableWriterAdapter.Builder extends Object
A builder to map key and value fields to table columns.

  • Constructor Details

    • Builder

      public Builder()

  • Method Details

    • kafkaPartitionColumnName

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder kafkaPartitionColumnName(@NotNull String kafkaPartitionColumnName)
      Set the name of the column which stores the Kafka partition identifier of the record.
      Parameters:
      kafkaPartitionColumnName - the name of the column in the output table
      Returns:
      this builder

    • offsetColumnName

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder offsetColumnName(@NotNull String offsetColumnName)
      Set the name of the column which stores the Kafka offset of the record.
      Parameters:
      offsetColumnName - the name of the column in the output table
      Returns:
      this builder

    • recvTimeColumnName

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder recvTimeColumnName(@NotNull String recvTimeColumnName)
      Set the name of the column which stores the time that the record was received by the ingester.
      Parameters:
      recvTimeColumnName - the name of the column in the output table
      Returns:
      this builder

    • timestampColumnName

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder timestampColumnName(@NotNull String timestampColumnName)
      Set the name of the column which stores the Kafka timestamp of the record.
      Parameters:
      timestampColumnName - the name of the column in the output table
      Returns:
      this builder

    • allowUnmapped

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder allowUnmapped(@NotNull String allowUnmapped)
      Allow the column with the given name to be unmapped in the output. Unmapped columns will have no setter, and will thus be null filled in the output.
      Parameters:
      allowUnmapped - the column name to allow to be unmapped
      Returns:
      this builder

    • addColumnToKeyField

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder addColumnToKeyField(@NotNull String column, @NotNull String field)
      Map a column to a field in the key record.
      Parameters:
      column - the name of the output column
      field - the name of the field in the key record
      Returns:
      this builder

    • addColumnToKey

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder addColumnToKey(@NotNull String column)
      Map a column to the key of the record. The key must match the type of the column. This configuration is intended for when the keys of your records are not in fact GenericRecords, but rather a primitive or a String.
      Parameters:
      column - the name of the output column
      Returns:
      this builder

    • addColumnToValueField

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder addColumnToValueField(@NotNull String column, @NotNull String field)
      Map a column to a field in the value record.
      Parameters:
      column - the name of the output column
      field - the name of the field in the value record
      Returns:
      this builder

    • addColumnToKeyFunction

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder addColumnToKeyFunction(@NotNull String column, @NotNull Function<org.apache.avro.generic.GenericRecord,?> function)
      Map a column to a function of the key record.
      Parameters:
      column - the name of the output column
      function - the function to apply to the key record
      Returns:
      this builder

    • addColumnToValueFunction

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder addColumnToValueFunction(@NotNull String column, @NotNull Function<org.apache.avro.generic.GenericRecord,?> function)
      Map a column to a function of the value record.
      Parameters:
      column - the name of the output column
      function - the function to apply to the value record
      Returns:
      this builder

    • allowNullKeys

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder allowNullKeys(boolean allowNullKeys)
      If allowNullKeys is set, then records with a null key will have their columns null filled; otherwise an exception is thrown on receipt of a null key. If no key fields are set, then no columns are taken from the value; so this flag has no effect.
      Parameters:
      allowNullKeys - if null key are allowed
      Returns:
      this builder

    • allowNullValues

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder allowNullValues(boolean allowNullValues)
      If allowNullValues is set, then records with a null value will have their columns null filled; otherwise an exception is thrown on receipt of a null value. If no value fields are set, then no columns are taken from the value; so this flag has no effect.
      Parameters:
      allowNullValues - if null values are allowed
      Returns:
      this builder

    • autoValueMapping

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder autoValueMapping(boolean autoValueMapping)
      If autoValueMapping is set, then all unused columns are mapped to a field of the same name in the generic record.
      Parameters:
      autoValueMapping - if unused columns should be automatically mapped to a field of the same name
      Returns:
      this builder

    • caseInsensitiveSearch

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder caseInsensitiveSearch(boolean caseInsensitiveSearch)
      If autoValueMapping(boolean) is set to true, then all unused columns are mapped to a field of the same name in the generic record without regard to case. For example, a generic record field of widgetId would be matched to a table column named WidgetID.
      Parameters:
      caseInsensitiveSearch - should a case-insensitive search for matching columns be performed?
      Returns:
      this builder

    • setFilter

      @ScriptApi @NotNull public GenericRecordConsumerRecordToTableWriterAdapter.Builder setFilter(Predicate<org.apache.kafka.clients.consumer.ConsumerRecord<?,?>> recordPredicate)
      If a filter is added, then all records will be passed through the predicate before being consumed. Only records that match the predicate will be ingested.
      Parameters:
      recordPredicate - returns true if the record should be ingested, otherwise false
      Returns:
      this builder