Package com.illumon.iris.logfilemanager

Class BinaryLogFileManager

java.lang.Object
com.illumon.iris.logfilemanager.BinaryLogFileManager
Direct Known Subclasses:
DateBinaryLogFileManager, StandardBinaryLogFileManager
public abstract class BinaryLogFileManager extends Object
Abstract class providing base logic and required methods to be implemented for classes which determine column partition information for binary log files. Any implementation must include a constructor which takes the same parameters as BinaryLogFileManager(Logger, DataRoutingService, LogtailerConfiguration.ConfigurationEntry, LogtailerMain.LogtailerRunDetails, LogtailerDestinationThrottles). That constructor should call this constructor with super(...).

  • Field Details

    • logDetails

      protected final String logDetails

  • Method Details

    • findAndSetMatchingFiles

      protected File[] findAndSetMatchingFiles(BinaryLogFileManager.TableLocationIdentifierStatus tableLocationIdentifierStatus)
      Regenerate and return the matching-files-array for the specified TableLocationIdentifierStatus by scanning the filesystem and replacing the list. The scan and the replacement are performed while holding BinaryLogFileManager.TableLocationIdentifierStatus.matchingFilesLock so that they cannot interleave with an incremental append in handleFile(java.nio.file.Path, boolean): the scan observes the filesystem and publishes its result atomically with respect to that append, so whichever runs second sees a consistent list. The lock is per-status, so this never blocks scans or appends for a different file set.
      Parameters:
      tableLocationIdentifierStatus - the tableLocationIdentifierStatus for the event that triggered this call
      Returns:
      the current array of matching files for this TableLocationIdentifierStatus

    • getTableLocationIdentifierStatusMap

      @VisibleForTesting public Map<TableIdentifier,Map<TableLocationIdentifier,BinaryLogFileManager.TableLocationIdentifierStatus>> getTableLocationIdentifierStatusMap()
      This is for testing only.
      Returns:
      the TableLocationIdentifier status map

    • handleFile

      public boolean handleFile(Path newFilePath, boolean isStartup)
      Given a new file, see if it belongs to this BinaryLogFileManager, and if so, handle it. This determination is based on the parsed TableLocationIdentifier for the file.
      Parameters:
      newFilePath - the new file's full Path
      isStartup - if true, this is a file found during startup
      Returns:
      true if the file was consumed by this BinaryLogFileManager

    • requestShutdown

      public void requestShutdown()
      Prevent any new destination threads from being created, and request shutdown on all the existing threads.

    • handleRoutingConfigChange

      public void handleRoutingConfigChange()
      Handle a routing configuration change by updating destinations for affected TableIdentifiers. This method detects which TableIdentifiers have changed destinations and updates only those, without restarting the entire BinaryLogFileManager.

      The statusMap updates are performed under synchronization, but the filesystem rescan is done outside the lock to avoid blocking directory watcher threads during potentially lengthy I/O.