AI Agent Quality Engineer

About ApexNeural
ApexNeural is a Hyderabad based enterprise AI studio building agentic systems for global clients. We are expanding our delivery team and are looking for quality engineers who can test both conventional software behaviour and the less deterministic behaviour of AI agents and agentic workflows.
About the role
You will own quality engineering for AI-enabled applications, with particular focus on testing AI agents, agent workflows, tool integrations, guardrails and LLM-generated outputs. The role combines conventional software and API testing with specialised testing and evaluation of agentic systems.
What you will do
• Design and execute test strategies for AI agents and agentic workflows.
• Test agent behaviour, tool selection, tool inputs and outputs, workflow execution and failure handling.
• Validate agent pipelines and orchestrated workflows, including different execution paths, retries, failures and edge cases.
• Test guardrails, permissions, policy enforcement and unsafe or unexpected agent behaviour.
• Evaluate LLM outputs for correctness, consistency, relevance, instruction following and safe failure behaviour.
• Develop test cases for hallucination, prompt injection, unexpected inputs, malformed tool responses and other adversarial scenarios.
• Validate API integrations and external service interactions used by agents.
• Analyse agent traces, logs and intermediate outputs to identify the source of failures.
• Build automated regression and evaluation suites for agent workflows and APIs.
• Establish repeatable evaluation criteria for AI outputs and track regressions across prompts, models and application changes.
• Perform conventional functional, integration, regression and API testing where required.
• Work with engineers to reproduce, diagnose and resolve defects.
Must have
• 2+ years of software QA or testing experience, preferably including automation.
• Strong understanding of API, integration and distributed-system testing.
• Strong test design skills including boundary, negative, adversarial and risk-based testing.
• Hands-on experience testing AI, LLM or agentic applications, or strong practical understanding of how such systems are built and evaluated.
• Ability to analyse agent execution, tool calls, intermediate results and final outputs.
• Python, JavaScript or TypeScript for test automation.
• Comfortable working with APIs, JSON, SQL, logs and basic Linux commands.
• Strong attention to edge cases and failure scenarios.
• Experience with Amazon Bedrock AgentCore and/or AWS Step Functions.
• Experience with agent frameworks, MCP or tool-calling systems.
• Experience with LLM evaluation or AI testing frameworks.
• Experience with Playwright, Pytest, Selenium, Postman or REST Assured.
• Security testing, prompt-injection testing or OWASP knowledge.

Good to have
• AWS or cloud experience.
• Experience with regulated or enterprise environments.