Skills mentioned in this posting (1)
AI
Sign in and add your skills to see which ones you're missing.
ABOUT THE ROLE
Join a remote expert network as a senior rater and reviewer, evaluating complex AI model outputs and helping calibrate other graders. Your work supports the development of high-quality, trustworthy AI systems across specialized fields.
WHAT YOU'LL DO
- Rank and rate expert-level model answers using detailed rubrics and write clear rationales.
- Evaluate steps in model reasoning across fields such as mathematics, coding, law, medicine, and finance.
- Fact-check complex claims against primary sources and verify accuracy.
- Assess answers against content policies, style guides, and edge cases.
- Review other raters' work, calibrate standards, and develop or refine grading guidelines.
- Assess search relevance and provide feedback in live sessions.
WHAT WE'RE LOOKING FOR
- At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or a relevant professional domain.
- Native or near-native written English proficiency.
- Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.
- Experience leading or reviewing other raters, such as as a QA lead, rubric owner, or reviewer of reviewers.
- An advanced degree, professional license, or deep working expertise in the domain you grade.
- Hands-on AI data annotation experience, search-quality rating, or experience writing and maintaining grader guidelines is a plus.
COMPENSATION & BENEFITS
Compensation is $50 to $120 per hour, depending on the project, field, and expertise level. Payment is based on hours worked.
LOCATION
Fully remote. Projects typically run from one week to several months.