Skip to content

Numerical type percent

Description

The numerical_type_percent check calculates the percentage of numerical values from column.

When to use

This sensor can be used when we receive a STRING column containing numerical values, and we would like to find the number of values that can be cast as NUMERICAL

Used sensor

Numerical type percent

Accepted rules

Min count

Count equals

Parameters

The sensor does not accept any parameters

How to use

The default configuration of column validity check numerical_type_percent on column id with min_count rule, looks like this 

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# yaml-language-server: $schema=https://cloud.dqo.ai/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
spec:
  target:
    schema_name: dqo_ai_test_data
    table_name: table
  time_series:
    mode: current_time
    time_gradient: day
  columns:
    id:
      type_snapshot:
        column_type: INT64
        nullable: true
      checks:
        validity:
          numerical_type_percent:
            rules:
              min_count:
                low:
                  min_value: 90.0
                medium:
                  min_value: 80.0
                high:
                  min_value: 70.0
The rendered query is 
SELECT
    CASE
        WHEN COUNT(analyzed_table.`id`)=0 THEN NULL
        ELSE 100.0*SUM(
                    CASE
                        WHEN SAFE_CAST( analyzed_table.`id` AS NUMERIC ) IS NOT NULL THEN 1
                        ELSE 0
                    END
                    )/COUNT(analyzed_table.`id`)
    END AS actual_value
FROM `dqo-ai-testing`.`dqo_ai_test_data`.`table` AS analyzed_table