Principal Data Engineer

We are seeking an experienced Principal Data Engineer with expertise in designing and leading enterprise-scale Data Engineering solutions. The ideal candidate should have strong hands-on experience in Snowflake, Databricks, Python, PySpark, SQL, ETL/ELT pipelines, Azure Data Factory, Airflow, Kafka, dbt, Azure Cloud, and Data Governance. This role involves architecting scalable cloud-native data platforms, leading engineering teams, implementing CI/CD standards, and delivering high-performance data solutions for enterprise finance and analytics.

Required Skills

Primary Skills

• Snowflake
• Databricks
• Python / PySpark
• Advanced SQL
• ETL / ELT Pipeline Development
• Azure Data Factory (ADF)
• Airflow
• Kafka / Event Hub
• dbt
• Azure Cloud (ADLS Gen2, Key Vault)
• Data Modeling
• CI/CD
• Terraform
• Data Governance

Experience

• 10+ years of Data Engineering experience.
• Strong expertise in Snowflake and Databricks platforms.
• Hands-on experience with Python, PySpark, SQL, ETL/ELT, and cloud-native data engineering.
• Experience designing enterprise-scale batch, streaming, and real-time data pipelines.
• Strong knowledge of Azure Cloud, Kafka, dbt, and orchestration frameworks.
• Experience leading engineering teams and enterprise data platform initiatives.
• FAANG / Product-Based / Tier-1 Company experience preferred.

Key Responsibilities

• Design and architect enterprise-scale ETL/ELT and streaming data pipelines.
• Lead implementation of Snowflake, Databricks, and cloud-native data platforms.
• Develop scalable Python/PySpark data processing frameworks.
• Build real-time data pipelines using Kafka, Event Hub, and Spark Structured Streaming.
• Establish engineering standards for CI/CD, testing, deployment, and automation.
• Implement enterprise data quality, governance, monitoring, and security frameworks.
• Optimize cloud infrastructure, storage, and compute performance.
• Design scalable enterprise data models and semantic layers.
• Mentor engineering teams and establish reusable engineering frameworks.
• Collaborate with architects, product managers, and business stakeholders to deliver enterprise data solutions.

Desired Competencies

• Strong analytical and problem-solving skills.
• Excellent leadership and mentoring abilities.
• Strong communication and stakeholder management skills.
• Experience architecting enterprise-scale data platforms.
• Deep understanding of cloud-native data engineering.
• Strong knowledge of finance and revenue data domains.
• Experience driving engineering best practices and platform modernization.

Key Deliverables

• Enterprise-scale Data Engineering solutions.
• ETL/ELT pipeline architecture and implementation.
• Snowflake and Databricks platform optimization.
• Streaming and real-time data pipelines.
• CI/CD implementation and automation.
• Data quality and governance frameworks.
• Cloud platform optimization.
• Engineering standards and reusable frameworks.

ABOUT BRICKRED SYSTEMS

BrickRed Systems is a global leader in next-generation technology consulting and workforce solutions, specializing in delivering high-quality talent across digital, engineering, marketing, analytics, finance, operations, and business transformation domains. With a strong emphasis on innovation, scalability, and client success, BrickRed Systems helps organizations solve complex business challenges by providing skilled professionals across strategy, technology, creative, and operational functions. BrickRed fosters a culture of continuous learning, collaboration, and excellence, enabling professionals to contribute to high-impact global initiatives while advancing their careers.