Senior Software Engineer – Python (LLM Evaluation & Repository Validation) | Job ID 37595

Vraify is hiring experienced Python software engineers for a remote contract focused on LLM evaluation and repository validation. The assignment is expected to last approximately 3 months; the start date is not yet confirmed.

Eligible locations
Applicants may be based in Bangladesh, Brazil, Colombia, Egypt, Ghana, India, Indonesia, Pakistan, Turkey, or Vietnam.

What you will do

• Analyze and triage issues in widely used public GitHub repositories.
• Set up and configure repositories, including Docker and development environment automation.
• Evaluate unit test coverage and quality, and run or modify code locally to assess AI-assisted bug fixing.
• Collaborate on selecting challenging repositories and issues; there may be opportunities to lead junior engineers.

What you bring

• Strong Python skills and proficiency with Git, Docker, and basic software pipeline setup.
• Ability to navigate complex codebases and run, modify, and test real-world projects locally.
• Experience with well-maintained, widely used public GitHub repositories, including repositories with 5,000+ stars.
• At least 2 years of software engineering experience and the ability to work at a technical lead level.
• Open-source contributions, LLM evaluation work, or developer-tool experience are helpful.

Engagement and pay

• Remote contractor assignment for approximately 3 months, 40 hours per week, with 4 hours of daily overlap with Pacific time.
• Hourly pay: $15 USD for 2–4 years of experience or $20 USD for 4+ years of experience. The exact rate is confirmed during screening.
• Contractor benefits do not include medical coverage or paid leave.
• Shortlisted applicants complete an online coding assessment.