Data Engineer Lead

Data Engineer Lead
Reports To: Data Platform Manager

Job Summary

• We are looking for a Data Warehouse Engineering Lead to develop and deploy data products through our Azure Analytics Platform. You will work closely with the DWH team, Data Architect and senior stakeholders across the business. It’s an exciting time to join us as we’re delivering our Data Strategy to support the transformation of our business through innovation & technology.

Main Duties & Responsibilities

• The role core duties include but are not limited to:
• Provide expertise and guidance to your direct reports ensuring that the team can deliver a vital Data Service to the business.
• Ensure our Data Governance principals are implemented and adhered to and to be an advocate of best practice in Data Product delivery.
• Build and maintain optimal data pipeline architecture.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc.
• Work with key stakeholders including Senior management, Developers, Data, and Project teams to assist with data-related technical issues and support their data infrastructure needs.
• Develop and maintain a group-wide Data Estate incorporating a Data Lake and Data Warehouse using best practice warehouse methodologies.
• Define data integration and ingestion strategies to ensure smooth and efficient data flow from
• various sources into the data lake and warehouse.
• Develop data modelling and schema design to support efficient data storage, retrieval, and analysis.

Key Skills

• Experience of leading a technical team working in a changing and fast paced environment.
• Experience with cloud platforms, particularly Azure, and a solid understanding of their data-related services and tools.
• Proficiency in SQL and one or more programming languages commonly used for data processing and analysis (e.g., Python, R, Scala).
• Familiarity with data integration techniques, ETL/ELT processes, and data pipeline frameworks.
• Knowledge of data governance, data security, and compliance practices.
• Strong analytical and problem-solving skills, with the ability to translate business requirements into scalable and efficient data solutions.
• Excellent communication and collaboration skills, with the ability to work effectively with cross- functional teams and stakeholders.
• Ability to adapt to a fast-paced and dynamic work environment and manage multiple priorities
• Strong analytic skills related to working with unstructured datasets.
• Strong project management and organizational skills.

Working relationships

• Collaborate closely with data architects and analysts to understand and fulfil their data requirements.
• Regular interaction with various departments to gather and provide data insights.

Communication:

• Communicate with leadership and colleagues in relation to all business activities
• Highly articulate and able to explain complex concepts in bite size chunks
• Strong ability to provide clear written reporting and analysis

Personal Qualities

• Ability to work to deadlines
• Strong problem-solving skills with an emphasis on product development.
• Excellent written and verbal communication skills.
• Adept at queries, report writing and presenting findings.

**Essential

• Cloud & Data Engineering Platforms: Azure Data Factory, Azure Databricks (including
• Unity Catalog & Notebooks), Azure Functions, Azure Apps, Azure DevOps Pipelines
• Data Transformation & Modelling: dbt (Data Build Tool) for scalable modelling and transformation in cloud data platforms
• Programming & Query Languages: PySpark, PySQL, T-SQL, MS SQL, Stored Procedures
• ETL/ELT & Data Integration: MS SSIS, WebHooks, RESTful APIs, automated pipelines for batch and streaming data
• Data Architecture & Quality: Data Modelling, Data Analysis, Data Quality frameworks, dimensional modelling
• Leadership & Communication: Proven experience in team management, mentoring cross-training initiatives, and coordinating with diverse business stakeholders
• Governance & Compliance: Data Privacy (GDPR, CCPA), Metadata management, data discovery, data security (RBAC, CLS, RLS)

*Desirable
*

• Infrastructure as Code & Automation: Terraform, CI/CD via Azure DevOps
• Real-Time Data Processing: Experience with Kafka, Spark Streaming, or Flink for streaming analytics
• Identity & Access Management: Entra ID (Azure AD) for role-based access control, group assignments, PIM management, and user provisioning
• Cloud Infrastructure & Operations: Deployment and monitoring of resources across Azure environments, with hands-on experience in cost optimization, resource governance, subscription management, and environment setup

Certifications:
Azure Administrator [AZ-104]
Fabric Data Engineer [DP-700]
Fabric Analytics Engineer [DP-600]
Azure DevOps Engineer [AZ-400]**