Data Quality Analyst (Databricks Platform)

Data Quality Analyst (Databricks Platform)
Reports To: Data Governance Lead

Job Summary

• We are seeking a proactive and detail‑oriented Data Quality Analyst to support the monitoring, investigation, and continual improvement of data quality across our Databricks platform.
• Using Databricks’ AI‑driven Data Quality Monitoring, Unity Catalog lineage, and Lakeflow observability, you will help detect issues early, interpret anomalies, and ensure our data remains accurate, trustworthy, and business-ready. Databricks’ agentic monitoring automatically learns expected data patterns and surfaces root causes via lineage and Lakeflow jobs.
• This role works closely with Data Governance, Data Engineering and Analytics teams and is ideal for someone with Databricks and SQL/Python skills who is passionate about data quality

Main Duties & Responsibilities

• Monitor Databricks Data Quality Monitoring alerts to identify anomalies, unexpected patterns, and potential data issues.
• Databricks’ monitoring uses AI agents to detect issues across the data estate at scale.
• Use Unity Catalog lineage and dataset certification tags to assess impact and prioritise issues.
• Investigate data quality incidents using Lakeflow jobs and pipeline observability to help identify root causes quickly.
• Assist in defining and maintaining data quality checks using SQL, Python, Delta Lake constraints, or DLT expectations.
• Support anomaly detection across key datasets, including schema drift, volume spikes/drops, late-
• arriving data, and distribution changes.
• These anomaly types are recommended components of robust DQ frameworks on Databricks.
• Work with Data Engineers to embed data quality checks and support automated monitoring within pipelines.
• Document data quality processes, checks, investigations, and repeatable remediation steps.
• Support Data Governance in reporting data quality metrics and driving continuous improvement efforts.

Key Skills

•  Hands-on experience with Databricks (notebooks, SQL, Python).
•  Strong SQL skills for profiling, querying, and validating data.
•  Understanding of data quality concepts (completeness, accuracy, timeliness, consistency).
•  Experience examining datasets to diagnose issues.
•  Ability to interpret quality alerts and conduct structured investigations.

Working relationships
The role works closely with:

•  Data Governance team
•  Data Engineers
•  Data Stewards
•  Analysts
•  Business domain stakeholders

Communication:

• Able to clearly communicate data quality issues, root causes, and impacts to both technical and non‑technical audiences.
• Comfortable presenting findings and producing structured documentation.
• Collaborates effectively across cross-functional teams to resolve issues and improve data quality standards.

Personal Qualities

•  Strong attention to detail with a passion for data accuracy and integrity.
•  Analytical thinker with the ability to investigate and interpret data issues.
•  Proactive, curious, and motivated to improve processes and data quality.
•  Committed to continuous learning in Databricks, data quality, and governance practices.

Knowledge / Key Skills:
**
Essential

•  Hands-on experience with Databricks (SQL and Python notebooks).
•  Strong SQL skills for data profiling, querying, and validating datasets.
•  Understanding of core data quality concepts (completeness, accuracy, timeliness, consistency).
•  Experience analysing datasets to diagnose anomalies or issues.
•  Familiarity with Delta Lake tables and basic data pipeline concepts.
•  Ability to interpret data quality alerts and perform structured investigations.

Desirable

• Experience with Unity Catalog lineage and metadata capabilities.
• Familiarity with Delta Live Tables (DLT) expectations and constraint-based checks.
• Exposure to anomaly detection or data validation frameworks.
• Knowledge of CI/CD and Git-based version control.
• Cloud or Databricks certifications.