Last updated: January 16, 2024

DQOps reference

The reference of DQOps file formats, data quality sensors and data quality rules is provided below.

Sensor and rule references

The following reference documentation shows the definition of data quality sensors and rules in DQOps.

• Data quality sensors reference

  The reference of data quality sensors shows the Jinja2 SQL templates used by data quality sensors for each data source.

• Data quality rules reference

  The reference of data quality rules shows the Python source code of data quality rules.

File format references

The following reference documentation describes YAML file formats and Parquet schema used by DQOps.

• DQOps YAML files reference

  All YAML files used by DQOps are described in this section. The most important files are:

  • .dqoconnection.yaml with the configuration of the data source.

  • .dqotable.yaml with the configuration of data quality checks for each table.

• Parquet file reference

  Describes the schema of Parquet files where DQOps stores data quality results.

Other references

The remaining reference documentation is found in other places.

• Data quality check reference lists all data quality checks supported by DQOps, showing also how to configure checks in DQOps YAML files, and what SQL is generated for each data quality check.

• REST API Python Client describes all REST API operations supported by DQOps, showing also examples of calling all operations from curl and from Python.