Package io.deephaven.enterprise.kafkawriter

Class KafkaTableWriter

java.lang.Object

io.deephaven.enterprise.kafkawriter.KafkaTableWriter

@ScriptApi
public class KafkaTableWriter
extends Object

The KafkaTableWriter creates a Community KafkaIngester that persists the data to disk and serves it to other queries using an in-worker instance of the DataImportServer.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	KafkaTableWriter.Options	A builder class containing the options for Kafka ingestion.

Constructor Summary

Constructors

Constructor	Description
KafkaTableWriter()

Method Summary

Modifier and Type	Method	Description
static void	consumeToDis(@NotNull KafkaTableWriter.Options options)	Consume a Kafka stream to a persistent Deephaven table.
static @NotNull io.deephaven.shadow.enterprise.com.illumon.iris.db.tables.dataimport.logtailer.DataImportServer	getDisByName(@NotNull String disName)	Retrieve a DataImportServer by name.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

KafkaTableWriter

public KafkaTableWriter()

Method Details

consumeToDis

@ScriptApi
public static void consumeToDis(@NotNull
 @NotNull KafkaTableWriter.Options options)

Consume a Kafka stream to a persistent Deephaven table.

The underlying ingestion is performed by the Deephaven Community KafkaTools class. In particular, the keySpec and valueSpec are Community KeyOrValueSpec objects.

Parameters:

options - an Options object that describes the ingestion process

getDisByName

@ScriptApi
@NotNull
public static @NotNull io.deephaven.shadow.enterprise.com.illumon.iris.db.tables.dataimport.logtailer.DataImportServer getDisByName(@NotNull
 @NotNull String disName)

Retrieve a DataImportServer by name.

The DIS is started as part of this call, and can be passed to the KafkaTableWriter.Options.dataImportServer(DataImportServer) function. The DIS can only be started one time, so if you are ingesting multiple tables each with a consumeToDis(Options) call, then you must call this function first and pass it to each Options structure.

Parameters:

disName - the name of the DIS to instantiate and start

Returns:

the dis referenced by the given name