Senior Software Engineer - Planner (E)

About e6data

The Lakehouse compute engine purpose-built for your heaviest, most compute-intensive workloads. Trusted by NASDAQ listed enterprises and leading unicorns to deliver 5x faster queries, over 50% Cost Savings, and Dismantling of ecosystem lock-in.

Enterprise data leaders are hit by an unprecedented explosion in compute demand for data intelligence. They are surprised to find that 10% of their heavy, compute-intensive use cases consume 80% of the cost, engineering effort and stakeholder complaints. Unfortunately, such workloads are also mission-critical and non-discretionary.

e6data amplifies ROI on enterprises' existing data platform and architecture - with Zero data movement, Zero application migration, and Zero Downtime.

e6data’s truly format-neutral compute has the unique distinction of being equally efficient and performant across leading Data Lakehouse table formats (Delta, Iceberg, Hudi), Data Catalogs (Unity, Hive, Glue, Fabric), Clouds (AWS, Azure, Google Cloud), Object stores (S3, ADLS Gen 2, GCS).

Our core team has 100+ years of combined experience and 10+ patents across diverse yet complementary disciplines like real-time databases, time-series databases, SQL OLAP engines, distributed computing platforms, stream processing, and log analytics. Two of the founding team are second-time founders with past exits. We’re all super proud to be building a deep tech product with impacting a multi-billion dollar market from right here in India!

Core Responsibilities

• Lead the development, testing, and maintenance of the query optimizer for e6data's data lake query engine.
• Implement advanced optimization techniques, such as cost-based optimization and rule-based optimization, to enhance engine performance.
• Design and build sophisticated capabilities in the optimizer, including the Learning Optimizer, which intelligently detects and generates essential statistics. Leverage past query executions to improve cardinality estimates.
• Fine-tune the optimizer to handle complex queries, such as those involving intricate joins, subqueries, or large datasets, ensuring the engine can support diverse customer workloads efficiently.
• Integrate AI-driven optimization techniques to significantly enhance the efficiency of SQL query execution.
• Mentor junior engineers and contribute to the overall architectural decisions of the engine team.

Must-Have Skills:

• Deep Understanding of Database Internals: Requires profound knowledge of how the query optimizer works, including an understanding of various access paths, join algorithms, and join ordering algorithms.
• Theoretical Understanding: A sound theoretical understanding of relational algebra and query rewrites.

Key Responsibilities:

• Problem-Solving Abilities: Exceptional ability to troubleshoot and optimize SQL execution plans, particularly in identifying and resolving performance bottlenecks.
• Programming Experience: Proficiency in Java, with extensive experience in building and optimizing large-scale distributed systems.
• Experience with Advanced Frameworks: Experience with Apache Calcite or similar query optimization frameworks; familiarity with AI and machine learning models, especially in the context of database systems.

Why Join Us?:

• Be part of a team that is pushing the boundaries of what’s possible in analytics.
• Lead initiatives that deliver tangible value and impact a wide array of industries.
• Enjoy a dynamic and innovative work environment where your contributions are recognized and valued.