Data Science Evaluator (contract)

We are seeking a Data Science Evaluator for a 12-week independent contractor engagement supporting the evaluation of AI systems performing real-world data science work. The role involves creating grading criteria, reviewing analyses and technical deliverables, documenting evidence-based scores, and applying consistent evaluation judgment. Candidates must have 2–3 years of professional data science experience, strong Python, SQL, statistics, machine learning, experimentation, and causal inference skills, plus excellent written English and the ability to work independently through ambiguity.

Apply here: https://app.scrini.ai/company/markethire/jobs/6aa38c77dd6f052491f95c8b

Key Responsibilities
Design task-specific grading criteria for exploratory analyses, statistical models, ML pipelines, A/B test write-ups, and technical notebooks.
Evaluate AI-generated and human-created data science work against established grading criteria.
Write clear, evidence-based justifications for every evaluation score.
Apply consistent evaluation judgment across varied technical deliverables and use cases.
Reason through ambiguity when assessing the quality, correctness, and completeness of submitted work.
Spot subtle errors in polished data science analyses and technical documentation.
Clearly separate established knowledge from inferences when documenting evaluation conclusions.
Iterate on evaluation work based on feedback from senior reviewers.
Follow complex instructions accurately while maintaining consistency across assigned evaluations.
Learn new tools quickly and work independently throughout the contract engagement.
Required Qualifications
2–3 years of professional data science experience.
Strong skills in Python, SQL, statistics, machine learning, experimentation, and causal inference.
Excellent written English.
Ability to follow complex instructions, reason through ambiguity, and spot subtle errors in polished work.
Ability to maintain a clear separation between what you know and what you are inferring.
Ability to work independently, learn new tools quickly, and take feedback well.
Based in and working from the US, Canada, or UK.
Available for 30–40+ hours per week for the full 12-week engagement.
Benefits and Perks
Fully remote contract engagement.
Competitive hourly rate of $100–$150.
Flexible scheduling within the required 30–40+ weekly hours.
Weekly payment via Stripe or Wise.
Potential for projects to be extended based on business needs and performance.
Opportunity to contribute to the evaluation of real-world AI data science capabilities for leading AI research organizations.