DATABRICKS ARCHITECT--AI

Domains : FinOps OR Healthcare OR PubSec Clearance etcKey Responsibilities:• Platform Architecture & Design: Lead the design and implementation of enterprise-scale, secure, and high-performance data architectures on theDatabricks Lakehouse Platform, leveraging Delta Lake, Unity Catalog, serverless compute, and advanced features like Photon engine.• Hands-On Implementation & Leadership: Provide expert hands-on guidance and lead development of complex data engineering pipelines using Apache Spark, PySpark, SQL, and Scala, with a focus on code quality, optimization, and best practices.• Performance Tuning & Optimization: Identify, diagnose, and resolve complex performance issues in large-scale data processing, query optimization, and cluster configurations.• Client Advisory & Strategy: Serve as the trusted senior technical advisor to client leadership, engineering, and data science teams, delivering best practices in data governance, security, MLOps, and operational excellence.• Knowledge Transfer & Team Enablement: Design and deliver advanced workshops, training sessions, and ongoing mentorship to accelerate client team proficiency and self-sufficiency.• Feature Adoption & Innovation: Champion the adoption of new Databricks features and tools (e.g., Photon, AI/BI, Mosaic AI) and integrate them effectively into client environments.• Cloud Integration: Ensure seamless, optimized, and secure integration with major cloud platforms (AWS, Azure, GCP) including storage, networking, IAM, and security services.Required Qualifications and SkillsTechnical Expertise:• 8+ years of experience in data engineering, big data architecture, or software engineering with a focus on data platforms.• 5+ years of deep, hands-on experience designing, implementing, and optimizing solutions on the Databricks platform.• Expert-level proficiency in Apache Spark ecosystem (PySpark, Spark SQL, Scala) and large-scale data processing.• Strong experience with cloud infrastructure (AWS, Azure, GCP), including storage solutions (S3, ADLS), networking, and security models.• Deep expertise in Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and best practices for ETL/ELT, data governance, and reliability.• Proficiency in Python, Scala, advanced SQL, and modern DevOps practices including Docker, Kubernetes, and CI/CD.Professional & Communication Skills:• Exceptional client-facing and presentation skills, able to communicate complex concepts to both technical and C-level audiences.• Proven leadership in client engagements, ability to drive consensus and influence decision-making across stakeholder groups.• Strong problem-solving, analytical skills, and a proactive, consultative mindset.Preferred Qualifications:• Databricks Certified Professional Certifications or Champion certifications.• Relevant cloud certifications (e.g., AWS Certified Big Data, Azure Data Engineer, Google Professional Data Engineer).• Experience with MLOps, generative AI, and advanced analytics on Databricks.• Prior experience in senior consulting, solution architecture, or resident roles