SWE-Bench Task Auditor (Remote US, Contract, $70–$90/hr) — Mercor via AI Gig Scout

Mercor is hiring SWE-Bench Task Auditors as contractors, open to candidates in the United States, at $70–$90/hr for 40 hours a week. You will evaluate the quality, correctness and reproducibility of software-engineering benchmark tasks used to train and evaluate a frontier AI lab's models: assessing repository-level tasks, reference patches, test harnesses and grading integrity, and giving clear, rubric-based written feedback.

Basic qualifications: 3+ years of professional software engineering; real open-source contribution or maintainer experience (merged PRs, committer or maintainer roles); a strong ability to audit reference patches, test runners and Docker isolation and to detect answer leakage or reward hacking; and fluency across common ecosystems (Python and at least one of Java, Go, TypeScript or C++). Preferred: familiarity with SWE-Bench (Verified) or similar repository benchmarks, maintainer history on major Python open-source projects (Django, Flask, scikit-learn, sympy, pytest and similar), and prior code-review or task-grading experience.

Details and apply link: https://aigigscout.com/jobs/mercor/swe-bench-task-auditor-list-aaaboepzuf806dv0eazav5rw?utm_source=kubernetesjobs&utm_medium=jobboard&utm_campaign=aigigscout_growth

Disclosure: posted by AI Gig Scout, an independent listing site (not Mercor). The apply link may be a referral link.