Package com.illumon.iris.validation.dynamic

Interface DynamicValidatorInterface

All Superinterfaces:
DataQualityTestCaseInterface
All Known Implementing Classes:
DynamicValidator
public interface DynamicValidatorInterface extends DataQualityTestCaseInterface
Interface containing all tests than can be executed during dynamic data validation. There is a contract that DynamicValidatorInterface does not have multiple methods with the same name.

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    assertAllValuesBetween(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue)
    Asserts that all values in the column are between [minValue,maxValue]
    void
    assertAllValuesEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
    Asserts that a column only contains a single value.
    void
    assertAllValuesEqualInputValue(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
    Asserts that a column only contains a single input value.
    void
    assertAllValuesInArrayInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
    Asserts that all values contained in arrays in a column are present in a set of expected values.
    void
    assertAllValuesInArrayNotInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
    Asserts that all values contained in arrays in a column are not present in a set of values.
    void
    assertAllValuesInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
    Asserts that all values in a column are present in a set of expected values.
    void
    assertAllValuesInStringSetInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
    Asserts that all values contained in string sets in a column are present in a set of expected values.
    void
    assertAllValuesInStringSetNotInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
    Asserts that all values contained in string sets in a column are not present in a set of values.
    void
    assertAllValuesNotEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
    Asserts that a column does not contain repeated values.
    void
    assertAllValuesNotEqualInputValue(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
    Asserts that a column does not contain the specified input value.
    void
    assertAllValuesNotInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
    Asserts that all values in a column are not present in a set of values.
    void
    assertAscending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically increasing values.
    void
    assertAvg(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
    Asserts that the average of the column is in the inclusive range [min,max].
    void
    assertColumnGrouped(String column)
    Asserts that a column is grouped.
    void
    assertColumnType(String column, Class type)
    Asserts that a column is of the expected type.
    void
    assertColumnTypes()
    Asserts that the column types in the table match the column types in the schema.
    void
    assertCountEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value1, Object value2)
    Asserts that a column contains the same number of rows for two given values.
    void
    assertDescending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically decreasing values.
    void
    assertEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are equal to all values in column2.
    void
    assertExpectedTableSize(int partitionsBefore, int partitionsAfter, double min, double max)
    Asserts the number of rows in the table is within a specified fraction of the expected size, determined by looking at the other tables in the database.
    void
    assertFracInf(boolean removeNull, boolean removeNaN, String column, double min, double max)
    Asserts that the fraction of infinite values is in the inclusive range [min,max].
    void
    assertFracNan(boolean removeNull, boolean removeInf, String column, double min, double max)
    Asserts that the fraction of NaN values is in the inclusive range [min,max].
    void
    assertFracNull(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
    Asserts that the fraction of NULL values is in the inclusive range [min,max].
    void
    assertFracValuesBetween(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue, double min, double max)
    Asserts that the fraction of values between [minValue,maxValue] is in the inclusive range [min,max].
    void
    assertFracWhere(String filter, double min, double max)
    Asserts the fraction of a table's rows matching the provided filter falls within a defined range.
    void
    assertFracZero(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
    Asserts that the fraction of zero values is in the inclusive range [min,max].
    void
    assertGreater(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are greater than all values in column2.
    void
    assertGreaterEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are greater than or equal to all values in column2.
    void
    assertLess(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are less than all values in column2.
    void
    assertLessEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are less than or equal to all values in column2.
    void
    assertMax(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
    Asserts that the maximum value of the column is in the inclusive range [min,max].
    void
    assertMin(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
    Asserts that the minimum value of the column is in the inclusive range [min,max].
    void
    assertNotEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are not equal to all values in column2.
    void
    assertNotNull(String... columns)
    Asserts that no null values exist in the specified columns.
    void
    assertNumberDistinctValues(boolean removeNull, boolean removeNaN, boolean removeInf, String columns, long min, long max)
    Asserts the number of distinct values is in the inclusive range [min,max].
    void
    assertPercentile(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double percentile, double min, double max, String... groupByColumns)
    Asserts that the defined percentile of the column is in the inclusive range [min,max].
    void
    assertSize(long min, long max)
    Asserts the number of rows in the table is in the inclusive range [min,max].
    void
    assertStd(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
    Asserts that the standard deviation of the column is in the inclusive range [min,max].
    void
    assertStrictlyAscending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically strictly increasing values.
    void
    assertStrictlyDescending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically strictly decreasing values.

    Methods inherited from interface com.illumon.iris.validation.DataQualityTestCaseInterface

    clearMessages, getMessages, message, setUp, tearDown

  • Method Details

    • assertSize

      void assertSize(long min, long max)
      Asserts the number of rows in the table is in the inclusive range [min,max].
      Parameters:
      min - minimum number of table rows
      max - maximum number of table rows
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertColumnType

      void assertColumnType(String column, Class type)
      Asserts that a column is of the expected type.
      Parameters:
      column - column to validate
      type - expected type
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertColumnGrouped

      void assertColumnGrouped(String column)
      Asserts that a column is grouped.
      Parameters:
      column - column to validate
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesEqual

      void assertAllValuesEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
      Asserts that a column only contains a single value.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesNotEqual

      void assertAllValuesNotEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
      Asserts that a column does not contain repeated values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesEqualInputValue

      void assertAllValuesEqualInputValue(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
      Asserts that a column only contains a single input value.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      value - make sure the column only contains this value
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesNotEqualInputValue

      void assertAllValuesNotEqualInputValue(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
      Asserts that a column does not contain the specified input value.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      value - make sure the column does not contain this value
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertEqual

      void assertEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertNotEqual

      void assertNotEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are not equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertLess

      void assertLess(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are less than all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertLessEqual

      void assertLessEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are less than or equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertGreater

      void assertGreater(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are greater than all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertGreaterEqual

      void assertGreaterEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are greater than or equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertNumberDistinctValues

      void assertNumberDistinctValues(boolean removeNull, boolean removeNaN, boolean removeInf, String columns, long min, long max)
      Asserts the number of distinct values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      columns - comma separated list of columns to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInDistinctSet

      void assertAllValuesInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
      Asserts that all values in a column are present in a set of expected values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      expectedValues - set of expected values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInArrayInDistinctSet

      void assertAllValuesInArrayInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
      Asserts that all values contained in arrays in a column are present in a set of expected values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      expectedValues - set of expected values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInStringSetInDistinctSet

      void assertAllValuesInStringSetInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
      Asserts that all values contained in string sets in a column are present in a set of expected values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      expectedValues - set of expected values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesNotInDistinctSet

      void assertAllValuesNotInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
      Asserts that all values in a column are not present in a set of values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      values - set of values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInArrayNotInDistinctSet

      void assertAllValuesInArrayNotInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
      Asserts that all values contained in arrays in a column are not present in a set of values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      values - set of values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInStringSetNotInDistinctSet

      void assertAllValuesInStringSetNotInDistinctSet(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
      Asserts that all values contained in string sets in a column are not present in a set of values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      values - set of values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracWhere

      void assertFracWhere(String filter, double min, double max)
      Asserts the fraction of a table's rows matching the provided filter falls within a defined range.
      Parameters:
      filter - filter
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracNull

      void assertFracNull(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
      Asserts that the fraction of NULL values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertNotNull

      void assertNotNull(String... columns)
      Asserts that no null values exist in the specified columns.

    • assertFracNan

      void assertFracNan(boolean removeNull, boolean removeInf, String column, double min, double max)
      Asserts that the fraction of NaN values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracInf

      void assertFracInf(boolean removeNull, boolean removeNaN, String column, double min, double max)
      Asserts that the fraction of infinite values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracZero

      void assertFracZero(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
      Asserts that the fraction of zero values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracValuesBetween

      void assertFracValuesBetween(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue, double min, double max)
      Asserts that the fraction of values between [minValue,maxValue] is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      minValue - minimum value for the value range
      maxValue - maximum value for the value range
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesBetween

      void assertAllValuesBetween(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue)
      Asserts that all values in the column are between [minValue,maxValue]
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      minValue - minimum value for the value range
      maxValue - maximum value for the value range
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertMin

      void assertMin(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
      Asserts that the minimum value of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertMax

      void assertMax(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
      Asserts that the maximum value of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAvg

      void assertAvg(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
      Asserts that the average of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertStd

      void assertStd(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
      Asserts that the standard deviation of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertPercentile

      void assertPercentile(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double percentile, double min, double max, String... groupByColumns)
      Asserts that the defined percentile of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      percentile - percentile of the column to test. Between 0 and 1.
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAscending

      void assertAscending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically increasing values. Consecutive values within a group must be equal or increasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertStrictlyAscending

      void assertStrictlyAscending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically strictly increasing values. Consecutive values within a group must be increasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertDescending

      void assertDescending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically decreasing values. Consecutive values within a group must be equal or decreasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertStrictlyDescending

      void assertStrictlyDescending(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically strictly decreasing values. Consecutive values within a group must be decreasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertExpectedTableSize

      void assertExpectedTableSize(int partitionsBefore, int partitionsAfter, double min, double max)
      Asserts the number of rows in the table is within a specified fraction of the expected size, determined by looking at the other tables in the database. For example min=0.8 and max=1.2 would assert that the table size is within 80% and 120% of the typical table size.
      Parameters:
      partitionsBefore - number of partitions before the current partition to compute expectation
      partitionsAfter - number of partitions after the current partition to compute expectation
      min - minimum fraction of expected rows in this table
      max - maximum fraction of expected rows in this table
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertCountEqual

      void assertCountEqual(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value1, Object value2)
      Asserts that a column contains the same number of rows for two given values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      value1 - make sure the column has the same number of value1 and value2 entries
      value2 - make sure the column has the same number of value1 and value2 entries
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertColumnTypes

      void assertColumnTypes()
      Asserts that the column types in the table match the column types in the schema.