Package com.illumon.iris.db.v2.locations.parquet

Class ParquetSchemaReader.ParquetMessageDefinition

java.lang.Object
com.illumon.iris.db.v2.locations.parquet.ParquetSchemaReader.ParquetMessageDefinition
Enclosing class:
ParquetSchemaReader
public static final class ParquetSchemaReader.ParquetMessageDefinition extends Object

  • Field Details

    • name

      public String name
      Yes you guessed right. This is the column name.

    • baseType

      public Class<?> baseType
      The parquet type.

    • wasWrittenByDH

      public boolean wasWrittenByDH
      An indicator of if the file was written by DH. We will not expose dictionary sources to the engine for files written by other services.

    • dhSpecialType

      public ColumnTypeInfo.SpecialType dhSpecialType
      Some types require special annotations to support regular parquet tools and efficient DH handling. Examples are StringSet and Vector; a parquet file with a IntVector special type metadata annotation, but storing types as repeated int, can be loaded both by other parquet tools and efficiently by DH.

    • noLogicalType

      public boolean noLogicalType
      Parquet 1.0 did not support logical types; if we encounter a type like this is true. For example, in parquet 1.0 binary columns with no annotation are used to represent strings. They are also used to represent other things that are not strings. Good luck, may the force be with you.

    • isArray

      public boolean isArray
      Your guess is good here

    • isGrouping

      public boolean isGrouping
      Your guess is good here.

    • codecType

      public String codecType
      When codec metadata is present (which will be returned as modified read instructions below for actual codec name and args), we expect codec type and component type to be present. When they are present, codecType and codecComponentType take precedence over any other DH type deducing heuristics (and thus baseType in this structure can be ignored).

    • codecComponentType

      public String codecComponentType

  • Constructor Details

    • ParquetMessageDefinition

      public ParquetMessageDefinition()