Databricks Solutions Architect

Databricks Resident Solutions Architect (RSA)

Location: 100% Remote Work

Duration: Long term(Open for Contract or Full Time)

Looking for a Databricks RSA who is a dedicated, post-sales technical leader embedded with client teams to design, implement, and optimize enterprise Lakehouse and AI architectures. Unlike pre-sales architects, an RSA focuses on long-term delivery, governance, code-level optimizations, and enablement.

Core Technical Requirements

• Databricks Lakehouse & Spark: Deep expertise in Apache Spark runtime internals, execution plan analysis (DAGs), memory tuning, Photon engine, and Delta Lake (ACID transactions, liquid clustering, vacuuming, data versioning).
• Languages: Advanced proficiency in PySpark and Python, with strong complex analytical SQL and optional Scala.
• Data Governance & Security: Hands-on setup of Unity Catalog (metastore architecture, data lineages, access controls, row/column-level masking, external storage credentials, system tables).
• Data Engineering & Orchestration: Production pipelines with Databricks Workflows, Lakeflow / Delta Live Tables (DLT), Auto Loader, and streaming architectures (Structured Streaming).
• Cloud Platforms: Production architecture experience across at least one major provider—AWS, Azure, or GCP—including VPC peering, IAM, Private Link, and cloud storage (S3, ADLS Gen2, GCS).
• DevOps & MLOps: Databricks Asset Bundles (DABs), CI/CD pipelines (GitHub Actions, Azure DevOps, or GitLab), Terraform for infrastructure as code, and MLflow / Mosaic AI serving.

Total Experience

• 8–12 years in data engineering, distributed systems, or enterprise solutions architecture.

Databricks Experience

• 3–5+ years of hands-on production design, deployment, and performance tuning.

Consulting Skills

Proven track record guiding cross-functional teams, running architectural review boards, and advising VP/C-suite stakeholders.

Certifications (High-Priority)

• Databricks: Databricks Certified Data Engineer Professional or Databricks Certified Solutions Architect (often mandatory or strongly preferred).
• Cloud: Cloud Architect/Data Engineer credentials (e.g., AWS Solutions Architect Professional, Azure Solutions Architect Expert, or GCP Professional Data Engineer).