Package com.illumon.iris.validation.dynamic

Interface DynamicValidatorInterface

All Superinterfaces:
DataQualityTestCaseInterface
All Known Implementing Classes:
DynamicValidator
public interface DynamicValidatorInterface
extends DataQualityTestCaseInterface
Interface containing all tests than can be executed during dynamic data validation. There is a contract that DynamicValidatorInterface does not have multiple methods with the same name.

  • Method Summary

    Modifier and Type Method Description
    void assertAllValuesBetween​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue)
    Asserts that all values in the column are between [minValue,maxValue]
    void assertAllValuesEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
    Asserts that a column only contains a single value.
    void assertAllValuesEqualInputValue​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
    Asserts that a column only contains a single input value.
    void assertAllValuesInArrayInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
    Asserts that all values contained in arrays in a column are present in a set of expected values.
    void assertAllValuesInArrayNotInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
    Asserts that all values contained in arrays in a column are not present in a set of values.
    void assertAllValuesInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
    Asserts that all values in a column are present in a set of expected values.
    void assertAllValuesInStringSetInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
    Asserts that all values contained in string sets in a column are present in a set of expected values.
    void assertAllValuesInStringSetNotInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
    Asserts that all values contained in string sets in a column are not present in a set of values.
    void assertAllValuesNotEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
    Asserts that a column does not contain repeated values.
    void assertAllValuesNotEqualInputValue​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
    Asserts that a column does not contain the specified input value.
    void assertAllValuesNotInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
    Asserts that all values in a column are not present in a set of values.
    void assertAscending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically increasing values.
    void assertAvg​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
    Asserts that the average of the column is in the inclusive range [min,max].
    void assertColumnGrouped​(String column)
    Asserts that a column is grouped.
    void assertColumnType​(String column, Class type)
    Asserts that a column is of the expected type.
    void assertColumnTypes()
    Asserts that the column types in the table match the column types in the schema.
    void assertCountEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value1, Object value2)
    Asserts that a column contains the same number of rows for two given values.
    void assertDescending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically decreasing values.
    void assertEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are equal to all values in column2.
    void assertExpectedTableSize​(int partitionsBefore, int partitionsAfter, double min, double max)
    Asserts the number of rows in the table is within a specified fraction of the expected size, determined by looking at the other tables in the database.
    void assertFracInf​(boolean removeNull, boolean removeNaN, String column, double min, double max)
    Asserts that the fraction of infinite values is in the inclusive range [min,max].
    void assertFracNan​(boolean removeNull, boolean removeInf, String column, double min, double max)
    Asserts that the fraction of NaN values is in the inclusive range [min,max].
    void assertFracNull​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
    Asserts that the fraction of NULL values is in the inclusive range [min,max].
    void assertFracValuesBetween​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue, double min, double max)
    Asserts that the fraction of values between [minValue,maxValue] is in the inclusive range [min,max].
    void assertFracWhere​(String filter, double min, double max)
    Asserts the fraction of a table's rows matching the provided filter falls within a defined range.
    void assertFracZero​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
    Asserts that the fraction of zero values is in the inclusive range [min,max].
    void assertGreater​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are greater than all values in column2.
    void assertGreaterEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are greater than or equal to all values in column2.
    void assertLess​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are less than all values in column2.
    void assertLessEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are less than or equal to all values in column2.
    void assertMax​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
    Asserts that the maximum value of the column is in the inclusive range [min,max].
    void assertMin​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
    Asserts that the minimum value of the column is in the inclusive range [min,max].
    void assertNotEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
    Asserts that all values in column1 are not equal to all values in column2.
    void assertNotNull​(String... columns)
    Asserts that no null values exist in the specified columns.
    void assertNumberDistinctValues​(boolean removeNull, boolean removeNaN, boolean removeInf, String columns, long min, long max)
    Asserts the number of distinct values is in the inclusive range [min,max].
    void assertPercentile​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double percentile, double min, double max, String... groupByColumns)
    Asserts that the defined percentile of the column is in the inclusive range [min,max].
    void assertSize​(long min, long max)
    Asserts the number of rows in the table is in the inclusive range [min,max].
    void assertStd​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
    Asserts that the standard deviation of the column is in the inclusive range [min,max].
    void assertStrictlyAscending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically strictly increasing values.
    void assertStrictlyDescending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
    Asserts that sub-groups of a column have monotonically strictly decreasing values.

    Methods inherited from interface com.illumon.iris.validation.DataQualityTestCaseInterface

    clearMessages, getMessages, message, setUp, tearDown

  • Method Details

    • assertSize

      void assertSize​(long min, long max)
      Asserts the number of rows in the table is in the inclusive range [min,max].
      Parameters:
      min - minimum number of table rows
      max - maximum number of table rows
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertColumnType

      void assertColumnType​(String column, Class type)
      Asserts that a column is of the expected type.
      Parameters:
      column - column to validate
      type - expected type
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertColumnGrouped

      void assertColumnGrouped​(String column)
      Asserts that a column is grouped.
      Parameters:
      column - column to validate
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesEqual

      void assertAllValuesEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
      Asserts that a column only contains a single value.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesNotEqual

      void assertAllValuesNotEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column)
      Asserts that a column does not contain repeated values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesEqualInputValue

      void assertAllValuesEqualInputValue​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
      Asserts that a column only contains a single input value.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      value - make sure the column only contains this value
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesNotEqualInputValue

      void assertAllValuesNotEqualInputValue​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value)
      Asserts that a column does not contain the specified input value.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      value - make sure the column does not contain this value
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertEqual

      void assertEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertNotEqual

      void assertNotEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are not equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertLess

      void assertLess​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are less than all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertLessEqual

      void assertLessEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are less than or equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertGreater

      void assertGreater​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are greater than all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertGreaterEqual

      void assertGreaterEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column1, String column2)
      Asserts that all values in column1 are greater than or equal to all values in column2.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column1 - column to test
      column2 - column to test
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertNumberDistinctValues

      void assertNumberDistinctValues​(boolean removeNull, boolean removeNaN, boolean removeInf, String columns, long min, long max)
      Asserts the number of distinct values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      columns - comma separated list of columns to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInDistinctSet

      void assertAllValuesInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
      Asserts that all values in a column are present in a set of expected values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      expectedValues - set of expected values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInArrayInDistinctSet

      void assertAllValuesInArrayInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
      Asserts that all values contained in arrays in a column are present in a set of expected values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      expectedValues - set of expected values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInStringSetInDistinctSet

      void assertAllValuesInStringSetInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... expectedValues)
      Asserts that all values contained in string sets in a column are present in a set of expected values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      expectedValues - set of expected values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesNotInDistinctSet

      void assertAllValuesNotInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
      Asserts that all values in a column are not present in a set of values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      values - set of values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInArrayNotInDistinctSet

      void assertAllValuesInArrayNotInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
      Asserts that all values contained in arrays in a column are not present in a set of values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      values - set of values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesInStringSetNotInDistinctSet

      void assertAllValuesInStringSetNotInDistinctSet​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object... values)
      Asserts that all values contained in string sets in a column are not present in a set of values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      values - set of values
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracWhere

      void assertFracWhere​(String filter, double min, double max)
      Asserts the fraction of a table's rows matching the provided filter falls within a defined range.
      Parameters:
      filter - filter
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracNull

      void assertFracNull​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
      Asserts that the fraction of NULL values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertNotNull

      void assertNotNull​(String... columns)
      Asserts that no null values exist in the specified columns.

    • assertFracNan

      void assertFracNan​(boolean removeNull, boolean removeInf, String column, double min, double max)
      Asserts that the fraction of NaN values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracInf

      void assertFracInf​(boolean removeNull, boolean removeNaN, String column, double min, double max)
      Asserts that the fraction of infinite values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracZero

      void assertFracZero​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max)
      Asserts that the fraction of zero values is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertFracValuesBetween

      void assertFracValuesBetween​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue, double min, double max)
      Asserts that the fraction of values between [minValue,maxValue] is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      minValue - minimum value for the value range
      maxValue - maximum value for the value range
      min - minimum fraction of values remaining after the filter. Between 0 and 1.
      max - maximum fraction of values remaining after the filter. Between 0 and 1.
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAllValuesBetween

      void assertAllValuesBetween​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable minValue, Comparable maxValue)
      Asserts that all values in the column are between [minValue,maxValue]
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      minValue - minimum value for the value range
      maxValue - maximum value for the value range
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertMin

      void assertMin​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
      Asserts that the minimum value of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertMax

      void assertMax​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Comparable min, Comparable max, String... groupByColumns)
      Asserts that the maximum value of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAvg

      void assertAvg​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
      Asserts that the average of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertStd

      void assertStd​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double min, double max, String... groupByColumns)
      Asserts that the standard deviation of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertPercentile

      void assertPercentile​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, double percentile, double min, double max, String... groupByColumns)
      Asserts that the defined percentile of the column is in the inclusive range [min,max].
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      percentile - percentile of the column to test. Between 0 and 1.
      min - minimum value for the value range
      max - maximum value for the value range
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertAscending

      void assertAscending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically increasing values. Consecutive values within a group must be equal or increasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertStrictlyAscending

      void assertStrictlyAscending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically strictly increasing values. Consecutive values within a group must be increasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertDescending

      void assertDescending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically decreasing values. Consecutive values within a group must be equal or decreasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertStrictlyDescending

      void assertStrictlyDescending​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, String... groupByColumns)
      Asserts that sub-groups of a column have monotonically strictly decreasing values. Consecutive values within a group must be decreasing.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to test
      groupByColumns - columns delineating groups for testing monotonicity
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertExpectedTableSize

      void assertExpectedTableSize​(int partitionsBefore, int partitionsAfter, double min, double max)
      Asserts the number of rows in the table is within a specified fraction of the expected size, determined by looking at the other tables in the database. For example min=0.8 and max=1.2 would assert that the table size is within 80% and 120% of the typical table size.
      Parameters:
      partitionsBefore - number of partitions before the current partition to compute expectation
      partitionsAfter - number of partitions after the current partition to compute expectation
      min - minimum fraction of expected rows in this table
      max - maximum fraction of expected rows in this table
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertCountEqual

      void assertCountEqual​(boolean removeNull, boolean removeNaN, boolean removeInf, String column, Object value1, Object value2)
      Asserts that a column contains the same number of rows for two given values.
      Parameters:
      removeNull - true to remove rows where column is NULL
      removeNaN - true to remove rows where column is NaN
      removeInf - true to remove rows where column is Inf
      column - column to validate
      value1 - make sure the column has the same number of value1 and value2 entries
      value2 - make sure the column has the same number of value1 and value2 entries
      Throws:
      DataQualityTestCase.AssertionFailed - the assertion failed

    • assertColumnTypes

      void assertColumnTypes()
      Asserts that the column types in the table match the column types in the schema.