Last updated: April 06, 2024

DQOps documentation guide

This page is the starting point to explore DQOps documentation. It lists all the main topics that you should learn to measure data quality with DQOps.

DQOps documentation

• What is DQOps

  ---

  DQOps is a data quality platform for data quality teams and data engineering teams to make data quality visible to business sponsors.

  Learn how DQOps addresses the needs for them.

  What is DQOps?

• Getting started

  ---

  Learn how to start using DQOps, install the platform locally, connect a data source, run a data quality check, and review the results on data quality dashboards.

  Getting started with DQOps

• DQOps concepts

  ---

  This section describes all the concepts behind DQOps. You will understand how data quality checks work and how to configure them.

  Learn about DQOps concepts

• Installation guide

  ---

  DQOps can be installed locally as a package downloaded from PyPI or as a Docker container. Learn how to install the platform.

  Download DQOps from PyPI or Docker Hub

• Data sources

  ---

  Find out what data sources DQOps supports and how to configure the connection parameters in the user interface or YAML files.

  List of supported data sources

• Categories of data quality checks

  ---

  Find out what types of common data quality issues are easy to detect with DQOps.

  Categories of data quality checks

• Data quality use cases and examples

  ---

  DQOps use cases and examples are step-by-step guides to solving the most common data quality issues.

  Review data quality use cases

• Working with DQOps

  ---

  Learn how to work with DQOps, performing regular activities such as reviewing data quality results or managing incidents.

  Working with DQOps

• DQOps Integrations

  ---

  Learn how to integrate DQOps with other data platforms or edit configuration files efficiently in Visual Studio Code.

  Integrating DQOps with other tools

References

• DQOps command-line interface

  ---

  The reference of all commands supported by the DQOps shell or usable from the command prompt.

  Command-line interface reference

• DQOps REST API Python Client

  ---

  Learn how to integrate DQOps directly in Python scripts. Run data quality checks from data pipelines or detect fatal issues with source tables.

  REST API Python Client reference

• Data quality checks reference

  ---

  The reference of all data quality checks provided in DQOps shows configuration examples in YAML and SQL queries that DQOps uses for each data source.

  Data quality checks reference

• Data quality sensors reference

  ---

  Data quality sensors are templates of SQL queries. Find out what SQL query DQOps uses for each data source.

  Data quality sensors reference

• Data quality rules reference

  ---

  Data quality rules are Python functions that evaluate data quality measures. Find out what rules are bundled in DQOps.

  Data quality rules reference

• DQOps YAML files reference

  ---

  DQOps stores the configuration of data sources and data quality checks in YAML files. Find the reference of every YAML element used by DQOps.

  DQOps YAML file schema reference

• DQOps parquet tables reference

  ---

  DQOps stores the data quality results in a Hive-compliant data lake. Find the schema reference of every Parquet table used by DQOps.

  Parquet data tables reference

What's more

You should start by reading what is DQOps to understand how DQOps can help you. Then follow the getting started guide.