Package io.deephaven.iceberg.base

Class IcebergUtils

java.lang.Object
io.deephaven.iceberg.base.IcebergUtils
public final class IcebergUtils extends Object

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final class 
    IcebergUtils.SpecAndSchema
    Used to hold a Schema, PartitionSpec and IcebergReadInstructions together.

  • Constructor Summary

    Constructors
    Constructor
    Description
    IcebergUtils()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    static Stream<org.apache.iceberg.DataFile>
    allDataFiles(@NotNull org.apache.iceberg.Table table, @NotNull org.apache.iceberg.Snapshot snapshot)
    Get a stream of all DataFile objects from the given Table and Snapshot.
    static Stream<org.apache.iceberg.ManifestFile>
    allManifestFiles(@NotNull org.apache.iceberg.Table table, @NotNull org.apache.iceberg.Snapshot snapshot)
    Get a stream of all ManifestFile objects from the given Table and Snapshot.
    static Type<?>
    convertToDHType(@NotNull org.apache.iceberg.types.Type icebergType)
    Convert an Iceberg data type to a Deephaven type.
    static org.apache.iceberg.types.Type
    convertToIcebergType(Class<?> columnType)
    Convert a Deephaven type to an Iceberg type.
    static boolean
    createNamespaceIfNotExists(@NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull org.apache.iceberg.catalog.Namespace namespace)
     
    static org.apache.iceberg.PartitionSpec
    createPartitionSpec(@NotNull org.apache.iceberg.Schema schema, @NotNull Iterable<String> partitionColumnNames)
     
    static IcebergUtils.SpecAndSchema
    createSpecAndSchema(@NotNull TableDefinition tableDefinition)
    Create PartitionSpec and Schema from a TableDefinition.
    static boolean
    dropNamespaceIfExists(@NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull org.apache.iceberg.catalog.Namespace namespace)
     
    static <T> Stream<T>
    toStream(org.apache.iceberg.io.CloseableIterable<T> iterable)
    Convert a CloseableIterable to a Stream that will close the iterable when the stream is closed.
    static void
    verifyPartitioningColumns(org.apache.iceberg.PartitionSpec tablePartitionSpec, TableDefinition tableDefinition)
    Check that all the partitioning columns from the partition spec are present in the Table Definition.
    static void
    verifyRequiredFields(org.apache.iceberg.Schema tableSchema, TableDefinition tableDefinition)
    Check that all required fields are present in the table definition

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • IcebergUtils

      public IcebergUtils()

  • Method Details

    • allDataFiles

      public static Stream<org.apache.iceberg.DataFile> allDataFiles(@NotNull @NotNull org.apache.iceberg.Table table, @NotNull @NotNull org.apache.iceberg.Snapshot snapshot)
      Get a stream of all DataFile objects from the given Table and Snapshot.
      Parameters:
      table - The Table to retrieve data files for.
      snapshot - The Snapshot to retrieve data files from.
      Returns:
      A stream of DataFile objects.

    • allManifestFiles

      public static Stream<org.apache.iceberg.ManifestFile> allManifestFiles(@NotNull @NotNull org.apache.iceberg.Table table, @NotNull @NotNull org.apache.iceberg.Snapshot snapshot)
      Get a stream of all ManifestFile objects from the given Table and Snapshot.
      Parameters:
      table - The Table to retrieve manifest files for.
      snapshot - The Snapshot to retrieve manifest files from.
      Returns:
      A stream of ManifestFile objects.

    • toStream

      public static <T> Stream<T> toStream(org.apache.iceberg.io.CloseableIterable<T> iterable)
      Convert a CloseableIterable to a Stream that will close the iterable when the stream is closed.

    • convertToDHType

      public static Type<?> convertToDHType(@NotNull @NotNull org.apache.iceberg.types.Type icebergType)
      Convert an Iceberg data type to a Deephaven type.
      Parameters:
      icebergType - The Iceberg data type to be converted.
      Returns:
      The converted Deephaven type.

    • convertToIcebergType

      public static org.apache.iceberg.types.Type convertToIcebergType(Class<?> columnType)
      Convert a Deephaven type to an Iceberg type.
      Parameters:
      columnType - The Deephaven type to be converted.
      Returns:
      The converted Iceberg type.

    • createSpecAndSchema

      public static IcebergUtils.SpecAndSchema createSpecAndSchema(@NotNull @NotNull TableDefinition tableDefinition)
      Create PartitionSpec and Schema from a TableDefinition.
      Returns:
      A IcebergUtils.SpecAndSchema object containing the partition spec and schema, and null for read instructions.

    • createPartitionSpec

      public static org.apache.iceberg.PartitionSpec createPartitionSpec(@NotNull @NotNull org.apache.iceberg.Schema schema, @NotNull @NotNull Iterable<String> partitionColumnNames)

    • createNamespaceIfNotExists

      public static boolean createNamespaceIfNotExists(@NotNull @NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull @NotNull org.apache.iceberg.catalog.Namespace namespace)

    • dropNamespaceIfExists

      public static boolean dropNamespaceIfExists(@NotNull @NotNull org.apache.iceberg.catalog.Catalog catalog, @NotNull @NotNull org.apache.iceberg.catalog.Namespace namespace)

    • verifyRequiredFields

      public static void verifyRequiredFields(org.apache.iceberg.Schema tableSchema, TableDefinition tableDefinition)
      Check that all required fields are present in the table definition

    • verifyPartitioningColumns

      public static void verifyPartitioningColumns(org.apache.iceberg.PartitionSpec tablePartitionSpec, TableDefinition tableDefinition)
      Check that all the partitioning columns from the partition spec are present in the Table Definition.