Package com.illumon.iris.db.v2.locations.local

Class LocalColumnLocation

java.lang.Object
com.illumon.iris.db.v2.locations.AbstractColumnLocation<LocalTableLocation>
com.illumon.iris.db.v2.locations.local.LocalColumnLocation
All Implemented Interfaces:
com.fishlib.base.log.LogOutputAppendable, StringUtils.StringKeyedObject, ColumnLocation<LocalTableLocation>, DeephavenFormatColumnLocation<LocalTableLocation>, NamedImplementation
public class LocalColumnLocation
extends AbstractColumnLocation<LocalTableLocation>
implements DeephavenFormatColumnLocation<LocalTableLocation>

Implements filesystem layout for Iris DB filesystem-backed columns, relative to a LocalTableLocation.

Header format explanations: (all sizes/offsets are in bytes)

Variables:

  • m = serialized metadata object size
  • d = data start position
  • m_h = min(d - 8, m) = size of the serialized metadata object prefix stored in the primary file's header region
  • m_o = m - m_h = size of the serialized metadata object suffix stored in an overflow file (usually 0)
  • e = end of file

Format for data and symbol offset files (by byte range):

  • [0 - 3]: serialized metadata object size (m)
  • [4 - 7]: data start position (d)
  • [8 - m_h): serialized metadata object prefix
  • [m_h - d): unused space
  • [d - e): binary column data

Note that m and d are written in big-endian byte order. The high bit of m denotes the byte order of all subsequent data in the file, with 0 denoting big-endian byte order and 1 denoting little-endian byte order. The higher 14 bits of d should also be considered reserved - the current maximum value used is 128K (1 << 17).

Typically, the metadata object for a data file is just the partitionKeysToRange map (a Map<TYPE, int[]>) or null if the column location is not grouped. For a symbol offset file, the metadata object is typically null.

Legacy column locations may have data and symbol offset files that instead contain an instance of a DiskMappedDataColumn subclass, which in turn may contain a valid partitionKeysToRange map. ColumnFileHeaderObjectInputStream handles translation in this case.