Package io.deephaven.enterprise.binlog.buffered

Interface Logger

All Superinterfaces:
AutoCloseable, Closeable, Flushable
All Known Implementing Classes:
MultiPartitionLoggerThreadSafeBase, MultiPartitionLoggerThreadUnsafeBase, PersistentQueryConfigurationLoggerImpl, PersistentQueryStateLoggerImpl, ProcessInfoFormat1Logger, QueryUserAssignmentLoggerImpl, ResourceUtilizationLoggerImpl, SinglePartitionLoggerThreadSafeBase, SinglePartitionLoggerThreadUnsafeBase
public interface Logger extends Flushable, Closeable
A buffered logger logs structured data into output bytes and flushes the encoded data to some output sink, potentially buffering multiple log calls before flushing to the output sink. An output sink in this context notifies failure via an IOException (as opposed to other logging strategies that may use a queue). As such, the output sink is typically backed by a physical sink such as a file or network which benefits from buffering.

Concrete implementations will provide a log method whose parameters define the structured data schema. The log method is responsible for encoding the data into the output bytes; and may internally perform a flush either because the logger has been configured to flush on every log, or because some other condition of the implementation has caused the log to flush (for example, an internal buffer has filled up).

Callers are encouraged to explicitly flush with a strategy that makes sense for their use-case and system setup. A few mock usages are shown below.

Basic usage: 


 try (MyLogger logger = MyLogger.of(outputSink)) {
     logger.log(timestamp1, value1, value2);
     logger.log(timestamp2, value3, value4);
     logger.flush();
 }

Advanced usage (logging task and flushing periodic action): 


 ScheduledExecutorService executor = ...;
 try (MyLogger logger = MyLogger.of(outputSink)) {
     Future<?> loggingFuture = executor.submit(() -> doLogging(logger));
     executor.scheduleWithFixedDelay(() -> doFlush(logger), 100, 100, TimeUnit.MILLISECONDS);
     // Block until doLogging is done
     loggingFuture.get();
     logger.flush();
 }

Advanced usage (queue): 


 BlockingQueue<Message> queue = ...;
 try (MyLogger logger = MyLogger.of(outputSink)) {
     while (!shutdown) {
         // timeout here ensures we re-check the shutdown condition
         Message message = queue.poll(1, TimeUnit.SECONDS);
         if (message == null) {
             continue;
         }
         // Consider the two extremes
         // 1. queue is almost always empty: this will log a small number of messages and then manually flush
         // 2. queue is almost always full: this will log in a continuous poll loop, flushing implicitly
         logger.log(message);
         while ((message = queue.poll()) != null) {
             logger.log(message);
         }
         logger.flush();
     }
 }

Use cases that are naturally partitioned from their upstream source should consider maintaining that partitioning with respect to loggers. In particular for high volume partitions, having a separate logger per upstream source is useful to reduce contention that would otherwise be introduced by using a shared logger.

Loggers are typically, but not always, thread-safe. In advanced scenarios where the caller is managing their own safety, thread-unsafe loggers may be used.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    close()
    Flushes any buffered data to the output sink and then closes this logger.
    void
    flush()
    Flushes any buffered data to the output sink.

  • Method Details

    • flush

      void flush() throws IOException
      Flushes any buffered data to the output sink.
      Specified by:
      flush in interface Flushable
      Throws:
      IOException - if an IO exception occurs

    • close

      void close() throws IOException
      Flushes any buffered data to the output sink and then closes this logger.

      If this logger has already been closed then this method returns immediately.

      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException - if an IO exception occurs