Package com.illumon.iris.binarystore

Class BinaryStoreWriterV2

java.lang.Object
com.illumon.iris.binarystore.AbstractBinaryStoreWriter<BinaryRowV2>
com.illumon.iris.binarystore.BinaryStoreWriterV2
All Implemented Interfaces:
Entry, Record, Row, TableWriter<BinaryRowV2>
Direct Known Subclasses:
BinaryStoreAggregatorWriterMultiPartition, BinaryStoreAggregatorWriterStandard
public class BinaryStoreWriterV2 extends AbstractBinaryStoreWriter<BinaryRowV2>
This class implements the writer for a self-descriptive binary table in the V2 binary log format.

  • Field Details

    • primaryRowWriter

      protected Row primaryRowWriter

    • primaryRecordWriter

      protected BinaryRecordV2 primaryRecordWriter

    • log

      protected final com.fishlib.io.logger.Logger log

  • Constructor Details

  • Method Details

    • computeHeaderBuffer

      public ByteBuffer computeHeaderBuffer() throws IOException
      Compute the header buffer for this writer and return it in a new buffer (direct). Callers of this method should cache the result if possible.
      Returns:
      a new buffer containing the binary header.
      Throws:
      IOException - pass through

    • makeCurrentDatePartitionedBinaryStoreWriter

      public static BinaryStoreWriterV2 makeCurrentDatePartitionedBinaryStoreWriter(com.fishlib.io.logger.Logger log, String logDir, String namespace, String tablename, TableIdentifier.NamespaceSet namespaceSet, String internalPartition, String[] columnNames, SupportedType[] columnTypes, Object[] typeMetadata, Map<String,Object> constantColumnValues, int applicationVersion, ZoneId zoneId) throws IOException
      Make a BinaryStoreWriterV2 instance that changes the column partition value based on the current time, by changing the name of the rolling output file.
      Parameters:
      log - the Logger instance
      logDir - the directory into which binary logs will be written
      namespace - the namespace
      tablename - the table name
      namespaceSet - the NamespaceSet
      internalPartition - the internal partition value
      columnNames - the names of the columns being logged
      columnTypes - the types of the columns being logged
      typeMetadata - the type metadata
      constantColumnValues - any constant column values
      applicationVersion - the application logger version
      zoneId - the time zone ZoneId
      Throws:
      IOException - from the initialization

    • makeSpecifiedPartitionBinaryStoreWriter

      public static BinaryStoreWriterV2 makeSpecifiedPartitionBinaryStoreWriter(com.fishlib.io.logger.Logger log, String logDir, String namespace, String tablename, TableIdentifier.NamespaceSet namespaceSet, String internalPartition, String columnPartition, String[] columnNames, SupportedType[] columnTypes, Object[] typeMetadata, Map<String,Object> constantColumnValues, int applicationVersion, ZoneId zoneId) throws IOException
      Make a BinaryStoreWriterV2 with the specified values, including an unchanging column partition value.
      Parameters:
      log - the Logger instance
      logDir - the directory in which the logs will be written
      namespace - the namespace (used to construct the file name)
      tablename - the table name (used to construct the file name)
      namespaceSet - the namespaceSet (used to construct the file name)
      internalPartition - the internal partition value (used to construct the file name)
      columnPartition - the column partition value (used to construct the file name)
      columnNames - the names of the columns being logged
      columnTypes - the types of the columns being logged
      typeMetadata - the type metadata
      constantColumnValues - any constant column values
      applicationVersion - the application logger version
      zoneId - the time zone, used to generate the date-time stamp at the end of the file name
      Returns:
      the BinaryStoreWriterV2 instance
      Throws:
      IOException - from the initialization

    • makeMultiPartitionFileManagerBinaryStoreWriter

      public static BinaryStoreWriterV2 makeMultiPartitionFileManagerBinaryStoreWriter(com.fishlib.io.logger.Logger log, String logDir, String namespace, String tablename, TableIdentifier.NamespaceSet namespaceSet, String internalPartition, String[] columnNames, SupportedType[] columnTypes, Object[] typeMetadata, Map<String,Object> constantColumnValues, int applicationVersion) throws IOException
      Make a BinaryStoreWriterV2 instance that uses dynamic (per-row) column partition values.
      Parameters:
      log - the Logger instance
      logDir - the directory into which binary logs will be written
      namespace - the namespace
      tablename - the table name
      namespaceSet - the NamespaceSet
      internalPartition - the internal partition value
      columnNames - the names of the columns being logged
      columnTypes - the types of the columns being logged
      typeMetadata - the type metadata
      constantColumnValues - any constant column values
      applicationVersion - the application logger version
      Throws:
      IOException - from the initialization

    • init

      protected void init() throws IOException
      Throws:
      IOException

    • getRowWriter

      public BinaryRowV2 getRowWriter()
      Description copied from interface: TableWriter
      Get a writer for a Row entries. This is likely to be newly created, so callers should cache this value. In practice, TableWriter implementations generally cache the result of the first call to this method as a primary writer.
      Returns:
      a Row, likely newly created

    • createBinaryRow

      protected BinaryRowV2 createBinaryRow(String[] columnNames, SupportedType[] columnTypes, Object[] typeMetadata)

    • getSetter

      public RowSetter getSetter(String name)
      Description copied from interface: TableWriter
      Gets a setter for a column.

      The implementation is likely to delegate to Row.getSetter(String) in a default Row instance.

      Parameters:
      name - column name
      Returns:
      setter for the column.

    • setFlags

      public void setFlags(Row.Flags flags)
      Description copied from interface: TableWriter

      The implementation is likely to delegate to Row.setFlags(Flags) in a default Row instance.

    • writeRow

      public void writeRow() throws IOException
      Description copied from interface: TableWriter
      Writes out a new row (values set using setters).

      The implementation is likely to delegate to Row.writeRow() in a default Row instance.

      Throws:
      IOException - problem writing the row

    • setColumnPartitionValue

      public void setColumnPartitionValue(String columnPartitionValue)
      Description copied from interface: Row
      For rows that are to be used with file managers that allow dynamic column partition selection, set the column partition value.
      Parameters:
      columnPartitionValue - the column partition value

    • getColumnPartitionValue

      public String getColumnPartitionValue()
      Description copied from interface: Row
      For rows that are to be used with file managers that allow dynamic column partition selection, retrieve the column partition value.
      Returns:
      the previously-set column partition value

    • writeDeferredRow

      public void writeDeferredRow(@NotNull WritableRowContainer<BinaryRowV2> row) throws IOException
      Description copied from interface: TableWriter

      Write the row to disk.

      Implementations of this method may choose to buffer rows for more efficient use of the underlying storage. Users should invoke TableWriter.flushDeferredRows() to tell the implementation to write any buffered rows.

      Parameters:
      row - The row to write (or potentially buffer)
      Throws:
      IOException - if an error occurs during write.

    • flushDeferredRows

      protected void flushDeferredRows(BinaryStoreWriterV2.DeferrableRowPartition part) throws IOException
      Flush all of the deferred rows in the input collection to the underlying stream.
      Parameters:
      part - The collection of deferred rows for the partitionVlaue
      Throws:
      IOException - if an error occurs during writing. Rows may be partially written, however there are no guarantees that the rows were written completely.

    • flushDeferredRows

      public void flushDeferredRows() throws IOException
      Description copied from interface: TableWriter
      Write any rows buffered by TableWriter.writeDeferredRow(WritableRowContainer) to storage.
      Throws:
      IOException - if an error occurs during write. Implementations that may write to more than one channel will write through to each channel and throw a MultiException wrapped in an IOException that contains the details on errors that occurred per channel.

    • writeMulti

      protected void writeMulti(ByteBuffer[] buffs, int count, GatheringByteChannel channel) throws IOException
      Perform a gathering write to the writer's channel. This method will not return until all buffers have been written or an error occurs.
      Parameters:
      buffs - The buffers to write
      count - The number of buffers to write.
      Throws:
      IOException

    • getColumnTypes

      public Class[] getColumnTypes()
      Description copied from interface: TableWriter
      Gets the column types for the table.
      Returns:
      column types for the table.

    • getColumnNames

      public String[] getColumnNames()
      Description copied from interface: TableWriter
      Gets the column names for the table.
      Returns:
      column names for the table.

    • setRecordData

      public void setRecordData(RecordData recordData)

    • getRecordSetter

      public <T extends RecordData> Record.RecordSetter<T> getRecordSetter(Class<T> tClass)
      Description copied from interface: TableWriter
      Gets a typed record setter for an entry.

      The implementation is likely to delegate to Record.getRecordSetter(Class) in a default Record instance.

      Returns:
      setter for the record.

    • getRecordSetter

      public Record.RecordSetter<RecordData> getRecordSetter()
      Description copied from interface: TableWriter
      Gets a record setter for an entry.

      The implementation is likely to delegate to Record.getRecordSetter() in a default Record instance.

      Returns:
      setter for the record.

    • writeRecord

      public void writeRecord(WritableByteChannel channel) throws IOException
      Description copied from interface: TableWriter
      Write a record entry to the provided output channel. The implementor is responsible for providing the data buffer.

      The TableWriter implementation is likely to delegate to Record.writeRecord(WritableByteChannel) in a default Record instance.

      Throws:
      IOException

    • getRecordWriter

      public BinaryRecordV2 getRecordWriter()
      Description copied from interface: TableWriter
      Get new writer for Record entries. This is likely to be newly created, so callers should cache this value. In practice, implementers generally cache the result of the first call to this method as a primary writer.
      Returns:
      a Record, likely newly created

    • createBinaryRecord

      protected BinaryRecordV2 createBinaryRecord()
      Implementation point for creating a binary record writer (BinaryRecordV2). Derived classes can override to change the details or type.
      Returns:
      a new BinaryRecordV2 record writer