Skills mentioned in this posting (1)
AI
Sign in and add your skills to see which ones you're missing.
ABOUT THE ROLE
Join a remote expert network as a senior rater and reviewer, evaluating challenging AI model outputs and helping calibrate other graders. Your work supports the development of reliable, high-quality AI systems.
WHAT YOU'LL DO
- Rank and rate expert-level model answers in your domain using detailed rubrics and written rationales.
- Evaluate model reasoning step by step in specialized fields such as mathematics, code, law, medicine, or finance.
- Fact-check detailed claims against primary sources and verify accuracy.
- Assess answers against content policies, style guides, and edge cases.
- Review other raters' work, calibrate standards, and develop or refine grading guidelines.
- Assess search relevance and share feedback in live sessions.
WHAT WE'RE LOOKING FOR
- At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or deep expertise in a relevant field.
- Native or near-native written English proficiency.
- Experience grading or reviewing expert-level answers against rubrics, policies, or style guides.
- Experience leading, reviewing, or mentoring other raters or reviewers.
- An advanced degree, professional license, or deep working expertise in the domain you evaluate.
- Hands-on experience with AI data annotation, preference ranking, rubric grading, process labeling, or search-quality rating is a plus.
COMPENSATION & BENEFITS
Compensation is $50 to $120 per hour, depending on project, field, and expertise. Payment is for hours worked.
LOCATION
Fully remote. Projects typically last from one week to several months.