Package io.deephaven.persistence

Class PersistenceBaseImpl<UpdateType,InputStreamType extends Closeable,OutputStreamType extends Flushable>

java.lang.Object
io.deephaven.persistence.PersistenceBaseImpl<UpdateType,InputStreamType,OutputStreamType>
All Implemented Interfaces:
Persistence<UpdateType>
Direct Known Subclasses:
PersistenceImpl
public class PersistenceBaseImpl<UpdateType,InputStreamType extends Closeable,OutputStreamType extends Flushable> extends Object implements Persistence<UpdateType>

A base implementation of the Persistence interface.

This implementation uses the following files to persist a collection and its updates.
  • A "checkpoint" file, created during each checkpoint commit, containing the entire collection at the time of the commit
  • Two "update" files, containing the commands since the last checkpoint commit, to be updated sequentially and swapped to ensure no corruption occurs
  • A "checkpoint id" file, containing the latest checkpoint number, which is named based on the persistenthashtable.checkpoint.idfile property
  • An "update version" file, containing which of the update files is current, which is named based on the persistenthashtable.checkpoint.updateversionfile property
All updates are written sequentially into two update files, with suffixes of _A and _B. The use of two files ensures that at least one is always valid; if a write error occurs, the most recent valid file will still be useful. When a checkpoint is created, the file names are of the form
  • (id)_(99999).ckp for the checkpoint file
  • (id)_(99999)(_A|_B).upd for the update files
where "id" represents the id of the collection (e.g. "PersistentQueryConfigurationsV2") and "99999" represents the commandId generated by the last invocation of the persist() method. The checkpoint file contains all of the elements of the collection as returned by Callbacks.getCheckpoint(), written in whatever order they are in in the List returned by getCheckpoint(). The update file with "99999" in it's name will contain streamed commands from commandId 99999 forward. When a persistent collection comes into existence, the "current" commandId is initialized to zero.

All files are created in the directory from the persistenthashtable.baseDirectory property value. This property allows dbroot, workspace, and devroot substitutions.

Immediate persisting of the UpdateType commands is done through the persist() method, which writes them to the update files. This method is called on every update.

The steps involved in checkpointing are:
  • 1 - the Persistence instance requests a *complete* copy of the current states of the in-memory collections (as a list of UpdateType object) via the Callbacks interface.
  • 2 - the Persistence instance creates new update files; any commands executed on the the collection while the checkpoint is being processed are written to these update files.
  • 3 - the list of commands (collection objects) returned by the in-memory collection are written to a new checkpoint file, and the checkpoint id file is updated with the new file's id
  • 4 - the old update files are closed, after which any new commands (updates) are sent to the update files created in step (2).

During initialization, this implementation reads the checkpoint id file, which contains a single string specifying the root names (without the .ckp or .upd extensions) of the most recently written checkpoint and update files. It also reads the update version file, which indicates which of the two versions file is the currently-used one.

Then the instance reads the corresponding checkpoint and update files, and passes the streamed instances to the Callbacks method handlePersistenceInitializationUpdate(). That is the only time that handlePersistenceInitializationUpdate() is ever called.

  • Field Details

  • Method Details

    • getId

      public String getId()
      Description copied from interface: Persistence
      Returns the collection ID that this persistence instance was constructed with.
      Specified by:
      getId in interface Persistence<UpdateType>
      Returns:
      the collection ID

    • cacheExists

      public boolean cacheExists()
      Description copied from interface: Persistence
      Returns true if a persisted image exists for this collection. The directory or directories in which the persisted files exist are left up to the implementation.
      Specified by:
      cacheExists in interface Persistence<UpdateType>
      Returns:
      true if an existing persisted collection exists

    • initPersistence

      public void initPersistence(boolean cleanFileOnError) throws IOException, ClassNotFoundException
      Description copied from interface: Persistence
      Initialize the Persistence instance by reading the persisted collection. At some point during the execution of this method, it is required that either the Callbacks implementation or the PersistenceStream implementation will call the setLastCommandId() method. If this does not happen, the commandIds will be reset to zero and the history of the collection on disk will be corrupted.
      Specified by:
      initPersistence in interface Persistence<UpdateType>
      Parameters:
      cleanFileOnError - if true, then clear the collection if an error occurs.
      Throws:
      IOException
      ClassNotFoundException

    • disconnect

      public void disconnect()
      Description copied from interface: Persistence
      Close the persistent collection. A checkpoint should be written if any command has been executed since initPersistence() or the last checkpoint commit (that is, if persist() has been called).
      Specified by:
      disconnect in interface Persistence<UpdateType>

    • disconnect

      public void disconnect(boolean forceCheckpoint)
      Description copied from interface: Persistence
      Behaves the same a disconnect() without arguments, but if forceCheckpoint is true a new checkpoint will be written regardless of whether any commands have been executed.
      Specified by:
      disconnect in interface Persistence<UpdateType>
      Parameters:
      forceCheckpoint - if true, force a checkpoint commit regardless of whether it is needed

    • persist

      public void persist(UpdateType update) throws IOException
      Description copied from interface: Persistence
      Persist the UpdateType object representing a command that has been executed on the collection.
      Specified by:
      persist in interface Persistence<UpdateType>
      Parameters:
      update - the command to be persisted
      Throws:
      IOException - from the persistence implementation

    • setLastCommandId

      public final void setLastCommandId(int cmdId)
      Description copied from interface: Persistence
      Must be called during the initPersistence() calls, so the commands executed following the return of that method begin with the correct commandId.
      Specified by:
      setLastCommandId in interface Persistence<UpdateType>
      Parameters:
      cmdId - the most recent command ID found when reading the previously persisted collection

    • nextCommandId

      public final int nextCommandId()
      Description copied from interface: Persistence
      Fetches the next commandId for a command that has been executed on the collection.
      Specified by:
      nextCommandId in interface Persistence<UpdateType>
      Returns:
      the next command ID