LLM Engineering Expert – AI Evaluation & Engineering Simulation

Role overview

Vraify, headquartered in Noida, India, is seeking experienced engineering professionals to create and evaluate challenging engineering-simulation tasks for AI systems. Design realistic, multi-constraint problems, validate simulations, and analyze AI/LLM outputs and failure modes.

Key responsibilities

• Create original engineering design and simulation problems with competing constraints and clear optimization goals.
• Build, run, and validate simulation environments using open-source tools and Python.
• Analyze AI/LLM and coding-agent outputs to identify reasoning, tool-use, and simulation failures.
• Develop reference solutions, objective graders, and evaluation criteria.
• Refine task difficulty based on model performance and collaborate with AI researchers and engineering/domain experts.

Requirements

• 10+ years of hands-on engineering design experience preferred.
• Background in Electrical, Mechanical, Aerospace, Control Systems, Systems, Robotics, or a related engineering discipline.
• Strong Python scripting skills and experience with at least one relevant open-source simulation tool: ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, or Gmsh.
• Strong understanding of engineering rigor, physical plausibility, units, boundary conditions, convergence, and technical documentation.
• Experience with LLMs, coding agents, AI evaluation, or analytical/QA work is a strong advantage.
• Reliable personal computer and high-speed internet for remote work. Weekend availability may be required.

Compensation & contract

• $500 per completed task (USD).
• Short-term contract, up to 24 weeks; immediate start.
• Availability: up to 40 hours/week, with 4 hours of overlap with PST.

Remote locations

Bangladesh, India, Indonesia, Egypt, Ghana, Kenya, Nigeria, Turkey, Brazil, Colombia, and all states in the USA.