Package io.deephaven.persistence

Interface Persistence<UpdateType>

All Known Implementing Classes:
PersistenceBaseImpl, PersistenceImpl
public interface Persistence<UpdateType>

This interface and the associated inner interfaces Callbacks and PersistenceStream specify a framework for streaming arbitrary collections of objects to disk, using a journaling and checkpoint strategy.

A persistent collection has an "id", which forms the root of the file names that the collection is streamed to on disk. For example, the persistent query controller uses the id "PersistentQueryConfigurationsV2" to store its persistent queries.

A persistent collection can be seen as a sequence of abstract "commands". The canonical commands for a collection are something like "put" and "remove", but the Persistence framework is not limited to these; it makes no assumptions whatsoever about the commands other that each command has a sequential integer "commandId", and that the current state of the collection can be represented by replaying the commands which created it in order. The checkpoint and update journals that are stored on disk are just encoded, sequential representations of the commands. The java type that corresponds to the commands are represented by the template parameter UpdateType.

As the commands (instances of UpdateType) are executed on the collection the current commandId is fetched and incremented by calling the nextCommandId() method.

The UpdateType instances represent the actual commands, which should be persisted immediately.

At specified intervals and in parallel with the execution and streaming of updates, the collection may be "checkpointed". At this time the implementation should ensure that its checkpoint file is complete, and reset its update files.

The only thing that existing persistence implementations themselves actually use the commandIds for is to generate unique names for the checkpoint and update files as the collection evolves.

Implementations will generally use two file types to ensure persistence. Checkpoint files (with the extension .ckp) store complete persistent collections. Update files (with the extension .upd) store all the commands performed between checkpoints. When restarted, the latest state of the collection can be recreated by reading the current checkpoint file, and replaying the current update file, through the initPersistence() implementation.

Sometime during the initPersistence() call, the method setLastCommandId() *must* be called by either the PersistenceStream implementation (which is responsible for opening and reading the actual disk files) or the Callbacks.handlePersistenceInitializationUpdate() method. This implies that the PersistenceStream implementation must store the correct value for the last commandId.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static interface 
    Persistence.Callbacks<UpdateType>
    Represents the Persistence instance's communication with the in-memory state of the collection.
    static interface 
    Persistence.PersistenceStream<UpdateType,InputStreamType extends Closeable,OutputStreamType extends Flushable>
    Represents the Persistence instance's communication with the on-disk representation of the collection.

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    cacheExists()
    Returns true if a persisted image exists for this collection.
    void
    disconnect()
    Close the persistent collection.
    void
    disconnect(boolean forceCheckpoint)
    Behaves the same a disconnect() without arguments, but if forceCheckpoint is true a new checkpoint will be written regardless of whether any commands have been executed.
    String
    getId()
    Returns the collection ID that this persistence instance was constructed with.
    void
    initPersistence(boolean cleanFileOnError)
    Initialize the Persistence instance by reading the persisted collection.
    int
    nextCommandId()
    Fetches the next commandId for a command that has been executed on the collection.
    void
    persist(UpdateType update)
    Persist the UpdateType object representing a command that has been executed on the collection.
    void
    setLastCommandId(int cmdId)
    Must be called during the initPersistence() calls, so the commands executed following the return of that method begin with the correct commandId.

  • Method Details

    • getId

      String getId()
      Returns the collection ID that this persistence instance was constructed with.
      Returns:
      the collection ID

    • cacheExists

      boolean cacheExists()
      Returns true if a persisted image exists for this collection. The directory or directories in which the persisted files exist are left up to the implementation.
      Returns:
      true if an existing persisted collection exists

    • initPersistence

      void initPersistence(boolean cleanFileOnError) throws IOException, ClassNotFoundException
      Initialize the Persistence instance by reading the persisted collection. At some point during the execution of this method, it is required that either the Callbacks implementation or the PersistenceStream implementation will call the setLastCommandId() method. If this does not happen, the commandIds will be reset to zero and the history of the collection on disk will be corrupted.
      Parameters:
      cleanFileOnError - if true, then clear the collection if an error occurs.
      Throws:
      IOException
      ClassNotFoundException

    • setLastCommandId

      void setLastCommandId(int cmdId)
      Must be called during the initPersistence() calls, so the commands executed following the return of that method begin with the correct commandId.
      Parameters:
      cmdId - the most recent command ID found when reading the previously persisted collection

    • nextCommandId

      int nextCommandId()
      Fetches the next commandId for a command that has been executed on the collection.
      Returns:
      the next command ID

    • persist

      void persist(UpdateType update) throws IOException
      Persist the UpdateType object representing a command that has been executed on the collection.
      Parameters:
      update - the command to be persisted
      Throws:
      IOException - from the persistence implementation

    • disconnect

      void disconnect()
      Close the persistent collection. A checkpoint should be written if any command has been executed since initPersistence() or the last checkpoint commit (that is, if persist() has been called).

    • disconnect

      void disconnect(boolean forceCheckpoint)
      Behaves the same a disconnect() without arguments, but if forceCheckpoint is true a new checkpoint will be written regardless of whether any commands have been executed.
      Parameters:
      forceCheckpoint - if true, force a checkpoint commit regardless of whether it is needed