ETL QA

About Us

Kumaran Systems: Engineering Future-Ready Transformations Since 1992, Kumaran Systems has been helping enterprises bridge the past and the future, reimagining legacy systems and embracing AI-led possibilities. With over 34 years of delivery excellence, we specialise in re-engineering core applications, driving cloud transformation, and building automation-first, GenAI-enabled ecosystems across the Banking and Financial Services, Insurance, Telecom, and Automotive sectors. We help organisations not just upgrade their systems but reimagine what’s possible.

Why Kumaran?

We blend engineering discipline with AI innovation, helping clients modernise with confidence, automate with clarity, and scale with purpose. Our global delivery model ensures agility, responsiveness, and seamless collaboration, with clients always at the heart of every engagement. At Kumaran, we don’t just solve problems, we engineer future-ready transformations.

Role Overview

We are looking for an ETL QA Engineer with strong experience in Databricks and PySpark, combined with hands-on exposure to AI-assisted development and prompt engineering. The ideal candidate will be responsible for validating large-scale data pipelines, ensuring data quality, and leveraging tools like GitHub Copilot, GitHub Actions/Runners, and AI agents to improve testing efficiency and coverage. You will also contribute to building and validating AI-powered knowledge bases.

Required Skills & Experience

• Core Technical Skills

• Strong hands-on experience in Databricks (notebooks, jobs, clusters, Delta Lake, SQL).

• Expert-level PySpark skills, including transformations, actions, window functions, and optimization techniques.

• Solid understanding of ETL/ELT concepts, data warehousing, and data modeling (star/snowflake schemas, dimension/fact tables).

• Proven experience in data validation, data quality checks, and reconciliation techniques.

• Practical experience with GitHub, GitHub Copilot, and GitHub Actions/Runners for CI/CD pipelines.

• AI & Prompt Engineering

• Experience crafting effective prompts for large language models (LLMs) for coding, testing, and documentation.

• Exposure to agent-based architectures or tools (e.g., workflow/agent frameworks that orchestrate multi-step AI tasks).

• Familiarity with AI-based tools for code analysis, test generation, or knowledge management.

• Knowledge Base & Documentation

• Experience building AI-driven knowledge bases or documentation systems (e.g., using vector search, embeddings, or LLM-based retrieval).

• Strong documentation skills, with the ability to translate complex data/QA concepts into clear knowledge articles.

• Testing & QA Practices

• Strong understanding of QA methodologies: test planning, test design, defect management, and regression testing.

• Experience with automated testing frameworks (Python-based testing libraries such as pytest , unittest , or similar).

• Familiarity with data quality tools/practices (e.g., validation rules, thresholds, anomaly detection).

Key Attributes

• Strong analytical and problem-solving skills with a detail-oriented mindset.
• Passion for data quality, automation, and continuous improvement.
• Ability to work in an agile, fast-paced environment and collaborate across multiple teams like Business and operations team and AD team.
• Curiosity and openness to adopting new AI tools and practices for QA.