Remote only
Not Available
About the job
Job Title: AI TesterLocation: Remote with occasional travel in Albany, NYStandard working hours - 9 hours/day$50 on C2CThe Medicaid Eligibility and Client Management (MECM) program is a multi-year modernization effort supporting the State Department of Health’s transformation of Medicaid eligibility and enrollment. As part of this transformation, the program is developing AI and GenAI capabilities to improve how members and staff interact with the system, streamline manual processes, and make enterprise knowledge easier to access.Role Overview Support testing and quality assurance of AI/GenAI solutions, including conversational AI, RAG, Agentic AI Validate AI solutions meet functional, business, and technical requirements.Key Responsibilities Develop and execute test plans, test cases, and evaluation scenarios for AI solutions Test AI outputs for accuracy, relevance, consistency, completeness, etc. Perform negative, edge-case, adversarial testing Validate content retention, guardrails, quality, etc. Track testing metrics and provide clear readiness recommendationsPreferred Qualifications Experience with software QA/testing; AI/ML or GenAI testing experience preferred Familiarity with LLMs and AI evaluation techniques Experience with test management / defect tracking tools (e.g., Jira) Strong analytical skills and ability to identify unexpected AI behavior Experience working in regulated or public-sector environments in a plus
