Senior AI Expert Rater and Reviewer (Remote - Worldwide)
Fully remote work; applicants must be able to work in English at a native or near-native level. A remote expert network is seeking senior raters and reviewers to evaluate challenging AI model outputs and help calibrate other graders. The work supports the development of reliable, high-quality AI systems. Projects typically run from one week to several months, and pay depends on the project, field and expertise, at roughly $50 to $120 per hour.
What you'll do
- Rank and rate expert-level model answers in your domain using detailed rubrics and written rationales.
- Evaluate model reasoning step by step in specialized fields such as mathematics, code, law, medicine or finance.
- Fact-check detailed claims against primary sources and verify their accuracy.
- Assess answers against content policies, style guides and edge cases.
- Review other raters' work, calibrate standards, and develop or refine grading guidelines.
- Assess search relevance and share feedback in live sessions.
What you need
- At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or deep expertise in a relevant field.
- Native or near-native written English proficiency.
- Experience grading or reviewing expert-level answers against rubrics, policies or style guides.
- Experience leading, reviewing or mentoring other raters or reviewers.
- An advanced degree, professional license, or deep working expertise in the domain you evaluate.
- Hands-on experience with AI data annotation, preference ranking, rubric grading, process labeling or search-quality rating is a plus.
Open to applicants anywhere in the world.
How to apply
Get the free i'llTip app to see the employer's details and apply. i'llTip's AI can tailor your résumé to this job, and you'll get alerts for similar roles.
Apply on i'llTipPrefer the website? Continue on illtip.comOr get the app: Android · iPhone
🔔 Get new jobs like this first: join Tech & AI Jobs on Telegram · or get personal job alerts
More Data Analysis/Entry jobs
Physical AI Data Collection Associate (Remote - Worldwide)
Remote projects are available, and some can be completed from home or your workplace using equipment provided by the employer. Some assignments require on-site work or travel. The posting does not list specific countries…
Performance Assurance Analyst, Federal Health IT (Remote - US only)
A technology services company that builds and supports mission-critical software for federal organizations is hiring a Performance Assurance Analyst for its federal healthcare technology work. The role independently meas…
Data Integration Engineer (Remote - US only)
S.-based applicants only. A technology services company that builds and supports solutions for federal organizations is hiring a data integration engineer for federal healthcare technology operations. The role centers on…
Data Analyst / Data Engineer (Remote, Egypt)
Working hours follow U.S. business hours. A recruitment firm is hiring on behalf of a client for a Data Analyst / Data Engineer. The role turns business data into reliable reporting, practical insights and scalable data …
P&ID Specialist (Remote - Worldwide)
Pay: $90/hr - $90/hr (hourly rate). US or Europe preferred, but location is flexible. A recruiting firm is sourcing specialists for a short, focused project supporting an AI developer. The work centers on reading, drawin…
Related job searches
Remote AI JobsAI jobsContract jobsSenior jobsWork From Home jobs in NigeriaWork From Home jobs