Senior Data Engineer

Simpalm is looking for one of its direct clients in the US, a fast-growing omnichannel Insurtech company, to fill a Lead Data Engineer role. This is an opportunity to lead the design and delivery of enterprise-scale, AI-ready data solutions that power analytics, machine learning, and Agentic AI across the organization.

The Lead Data Engineer will own the design, development, and delivery of complex data engineering solutions, partnering closely with data engineers, architects, data scientists, and AI engineers to translate business needs into scalable data and AI platforms, including support for LLMs, Retrieval-Augmented Generation (RAG), and agentic AI frameworks.

KEY RESPONSIBILITIES:

• Lead the analysis, design, development, and implementation of scalable data solutions within the enterprise data warehouse and modern data platform.
• Partner with Product Engineers, Data Engineers, Architects, Data Scientists, and AI Engineers to translate business requirements into scalable data and AI solutions.
• Design, build, and optimize ETL/ELT pipelines using Snowflake, Nexla, SQL, and Python.
• Develop and manage Apache Airflow DAGs and job scheduling for reliable, observable data processing.
• Design AI-ready data architectures supporting analytics, machine learning, GenAI, RAG, vector databases, and Agentic AI solutions.
• Leverage AI-assisted development tools (Microsoft Copilot, Claude, Cursor) to accelerate delivery and improve code quality and documentation.
• Define data ingestion, transformation, governance, and serving patterns for AI model training and inference workloads.
• Support metadata management, data lineage, data observability, and governance practices for enterprise AI initiatives.
• Evaluate and recommend emerging AI technologies, frameworks, and best practices to improve engineering capability.
• Manage agile sprint priorities, backlog refinement, story estimation, dependency tracking, and delivery commitments.
• Coordinate across onsite, offsite, offshore, and cross-functional teams to resolve blockers and maintain delivery momentum.
• Create and maintain technical design docs, data flow diagrams, AI solution architecture diagrams, and operational runbooks.
• Drive data quality, performance tuning, monitoring, and production support for critical pipelines and Snowflake workloads.
• Lead architecture reviews, code reviews, root-cause analysis, and problem-solving sessions with engineering and business partners.
• Mentor team members on modern data engineering and AI-assisted development practices.
• Promote responsible AI practices on data privacy, security, governance, and ethical use of AI.

MUST-HAVE SKILLS & EXPERIENCE:

• Bachelor's degree in computer science, IT, engineering, data science, AI, or a related field or equivalent experience.
• Strong hands-on Snowflake experience: SQL development, performance tuning, data modeling, stored procedures, scalable processing patterns.
• Hands-on ETL/ELT pipeline experience with Nexla and related modern data integration tools.
• Experience with Apache Airflow (or a similar orchestration tool), DAG development, scheduling, monitoring, and troubleshooting.
• Strong proficiency in SQL and Python for enterprise-scale data engineering.
• Practical experience with AI-powered dev tools, Microsoft Copilot, Claude, Cursor, GitHub Copilot, or similar.
• Working knowledge of GenAI concepts: LLMs, RAG, vector embeddings, prompt engineering, and agentic AI patterns.
• Experience designing data architectures/pipelines supporting AI, ML, analytics, and intelligent automation.
• Knowledge of data governance, security, privacy, lineage, and compliance for AI and enterprise data platforms.
• Proven ability to lead sprint execution, manage competing priorities, clarify scope, identify risks, and coordinate delivery.
• Strong collaboration across product engineering, data engineering, AI/ML, BI/analytics, QA, architecture, and business teams.
• Experience working with onsite, offsite, offshore, and distributed teams in an Agile environment.
• Strong analytical, problem-solving, communication, and troubleshooting skills.
• Self-starter with strong ownership, accountability, and attention to detail in a fast-paced environment.
• Demonstrated ability to learn and adopt emerging AI technologies in a rapidly evolving landscape.

PREFERRED (NICE TO HAVE):

• Experience with Snowflake on Azure and related Azure data services.
• Experience with Azure OpenAI, Microsoft Fabric, Databricks, Amazon Bedrock, Anthropic Claude, OpenAI, or similar AI platforms.
• Experience implementing RAG, semantic search, vector databases, or agentic AI frameworks (LangChain, LangGraph, CrewAI, AutoGen, Semantic Kernel).
• Experience building AI-enabled data products, intelligent document processing, conversational AI, or enterprise knowledge retrieval systems.
• Experience with additional ETL/ELT tools: Azure Data Factory, Talend, and Python-based frameworks.
• Experience with DevOps, CI/CD, Infrastructure-as-Code, and MLOps supporting AI/data platform deployments.
• Experience with Jira, Azure DevOps, GitHub, or similar Agile delivery/tracking tools.
• Experience with metadata management, data quality frameworks, or Master Data Management (MDM) initiatives.
• Experience in insurance, insurtech, healthcare, or financial services.
• Relevant certifications in Snowflake, Azure, Generative AI, Claude, Microsoft Copilot, AI engineering, or cloud data platforms.