Skip to content

Distinct count percent

Description

The distinct_count_percent calculates the percentage of unique values.

When to use

This is a standard quality check, mostly used when we would like to simply ensure that columns contain only unique values.

Used template

Distinct count percent

Accepted rules

Min count

Count equals

Parameters

This check takes no parameters.

How to use

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# yaml-language-server: $schema=https://cloud.dqo.ai/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
spec:
  target:
    schema_name: dqo_ai_test_data
    table_name: table
  time_series:
    mode: current_time
    time_gradient: day
  columns:
    countries:
      type_snapshot:
        column_type: STRING
        nullable: true
      checks:
        uniqueness:
          distinct_count_percent:
            rules:
              min_count:
                low:
                  min_value: 90.0
                medium:
                  min_value: 80.0
                high:
                  min_value: 70.0

SELECT
    (count(distinct analyzed_table.`countries`) / count(analyzed_table.`countries`)) * 100 AS actual_value
FROM `dqo-ai-testing`.`dqo_ai_test_data`.`table` AS analyzed_table