Senior Software Engineer – Ruby (LLM Evaluation & Repository Validation) | Job ID 37600

Vraify is hiring experienced Ruby software engineers for a remote contractor assignment focused on LLM evaluation and repository validation. The engagement is expected to last approximately 3 months; the start date is to be confirmed.

Eligible locations
Applicants may be based in Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Pakistan, Turkey, or Vietnam.

What you will do

• Analyze and triage issues in public GitHub repositories.
• Set up repositories and development environments, including Docker.
• Evaluate unit test coverage and quality, and run or modify code locally to assess AI-assisted bug fixing.
• Collaborate with researchers on challenging software engineering tasks; there may be opportunities to lead junior engineers.

What you bring

• At least 3 years of strong Ruby experience and the ability to contribute at a technical lead level.
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to navigate, run, modify, and test complex real-world codebases.
• Experience with well-maintained, widely used public GitHub repositories, including repositories with 5,000+ stars.
• Open-source contributions, LLM evaluation, developer tools, or automation-agent experience are helpful.

Engagement and pay

• Remote contractor assignment for approximately 3 months, 40 hours per week, with 4 hours of daily overlap with Pacific time.
• Hourly pay: $15 USD for 2–4 years of experience or $20 USD for 4+ years of experience. The exact rate is confirmed during screening.
• Contractor benefits do not include medical coverage or paid leave.
• Shortlisted applicants complete an online coding assessment.