Senior/Lead Data Engineer (Databricks)

Senior/Lead Data Engineer (Databricks)

Experience: 8-14 Years

Location: Remote

Salary: Negotiatble

Job Summary

We are looking for an experienced Databricks and Data Mesh Technical Lead to provide

technical leadership across the Databricks and Data Mesh ecosystem. The role is

responsible for translating business and Data Product requirements into scalable technical

solutions, driving architecture and delivery, strengthening Data Mesh governance, and

ensuring platform security, performance, cost optimization, and operational excellence. The

ideal candidate should have strong hands-on experience with Databricks, Data Mesh, Unity

Catalog, data governance, cloud data platforms, and technical leadership.

Key Responsibilities

• Provide technical leadership across the Databricks and Data Mesh ecosystem.
• Translate business and Data Product requirements into scalable solutions and delivery roadmaps.
• Own the technical backlog, architecture, delivery, enhancements, defects, and technical debt.
• Lead the development and improvement of Data Mesh governance and self-service tools.
• Ensure compliance with Unity Catalog, security, least-privilege access, governance, and audit requirements.
• Own Usage and Cost Dashboards, ensuring accurate and actionable insights.
• Communicate technical options, dependencies, risks, and constraints to stakeholders.
• Support production operations, escalated incidents, and service improvements.
• Lead engineers through technical guidance, performance feedback, and best practices.
• Manage project scope, resources, schedules, budgets, and delivery.
• Use platform metrics, cost insights, and stakeholder feedback to continuously improve Data Mesh capabilities.

Must-Have Skills

• Strong software and solution-engineering experience in cloud data-platform applications, APIs, automation, and self-service capabilities.
• Experience with technical architecture and scalable application delivery.
• Working knowledge of Databricks and modern data-engineering practices.
• Understanding of Data Mesh, Unity Catalog, data governance, RBAC, ABAC, security, and auditability.
• Experience with cloud platforms, DevOps, monitoring, and production support.
• Strong technical leadership, problem-solving, stakeholder-management, and communication skills.
• Experience leading engineering teams and managing multiple projects.
• Knowledge of Agile and Waterfall delivery methodologies.

Preferred Skills

• Databricks Data + AI Platform Administration
• Terraform or Infrastructure as Code
• Analytics and AI/ML support
• LLM and Generative AI/ Agentic AI knowledge
• OpenAI Codex or Anthropic Claude Code

Certifications

• AWS Platform Architect

• Azure Platform Architect

• Databricks Data Engineer Professional