Hires remotely in
Remote Work Policy
Remote only
Company Location
Prosper
Visa Sponsorship
Not Available
RelocationAllowed
Skills
Python
SQL
Networking
Security
Identity Management
Cloud Architecture
Azure
Monitoring
Data Governance
Data Quality
Apache Spark
AWS
Metadata Management
Pyspark
Infrastructure as code
data lineage
GCP
databricks
CI/CD
Delta Lake
Production Deployment
Access Controls
Platform Automation
Databricks Asset Bundles
About the job
Who We Are
Artmac Soft is a technology consulting and service-oriented IT company dedicated to providing innovative technology solutions and services to customers.
Job Description
Job Title : Lead Databricks Engineer
Job Type : C2C / W2
Experience : 8-15 Years
Location : Remote
Role Summary
We are seeking an experienced Lead Databricks Engineer to lead the design, development, and implementation of scalable, high-performance data engineering solutions using the Databricks Lakehouse Platform. The ideal candidate will possess strong hands-on expertise in Databricks, Apache Spark, PySpark, SQL, Delta Lake, and modern data architecture.
Key Responsibilities
- Own the technical roadmap for Databricks-based data engineering initiatives, aligning platform architecture with enterprise data strategy and business goals.
- Lead architecture reviews and make high-impact technical decisions involving data platform design, technology selection, scalability, and maintainability.
- Define reusable engineering frameworks, reference architectures, coding standards, and development guidelines across multiple data engineering projects.
- Drive the implementation of domain-oriented data architecture and data product strategies to improve data accessibility, ownership, and reuse.
- Establish architectural patterns for multi-environment data platforms, including development, testing, staging, and production.
- Lead the adoption of Infrastructure as Code and automated platform provisioning to standardize Databricks environment management.
- Partner with security, governance, and compliance teams to establish enterprise-wide policies for sensitive data, auditability, and regulatory requirements.
- Manage technical dependencies, delivery risks, and architectural trade-offs across multiple teams and concurrent data initiatives.
Required Skills And Experience
- 8 15 years of experience in designing and delivering enterprise data platforms using Databricks and cloud-native technologies.
- Strong proficiency in Apache Spark, PySpark, Python, SQL, and distributed data processing.
- Expertise in lakehouse architecture, Delta Lake, data architecture patterns, and large-scale data platform design.
- Proven experience establishing data governance, metadata management, access controls, and data lineage practices.
- Strong understanding of cloud architecture, networking, identity management, and security across Azure, AWS, or GCP.
- Experience with platform automation, Infrastructure as Code, CI/CD, and production deployment strategies.
- Ability to evaluate complex technical requirements and translate them into scalable architectural solutions.
- Knowledge of Databricks Asset Bundles and automated deployment practices.
- Understanding of data governance, security, data quality, and monitoring.
Qualification
- Bachelor's degree or equivalent combination of education and experience.
About the company
Learn more about Artmacsoft private limited 
