Skip to content

Distinct count

Description

The distinct_count sensor counts unique values in a specified column.

When to use

This is a standard quality check, mostly used when we would like to simply ensure that columns contain only unique values.

Used template

Distinct count

Accepted rules

Min count

Count equals

Parameters

This check takes no parameters.

How to use

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# yaml-language-server: $schema=https://cloud.dqo.ai/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
spec:
  target:
    schema_name: dqo_ai_test_data
    table_name: table
  time_series:
    mode: current_time
    time_gradient: day
  columns:
    countries:
      type_snapshot:
        column_type: STRING
        nullable: true
      checks:
        uniqueness:
          distinct_count:
            rules:
              min_count:
                low:
                  min_value: 90.0
                medium:
                  min_value: 80.0
                high:
                  min_value: 70.0

SELECT
    count(distinct analyzed_table.`countries`) AS actual_value
FROM `dqo-ai-testing`.`dqo_ai_test_data`.`table` AS analyzed_table