Senior AI Rater and Reviewer (Remote - Worldwide)
Pay: $50/hr - $120/hr (hourly rate). This is a remote, project-based opportunity with an expert network that trains and evaluates AI systems. Senior raters grade complex model outputs against detailed rubrics and also help calibrate the other graders on the team. Projects run from about one week to several months, and pay depends on the project, subject area and level of expertise.
What you'll do
- Rank and rate expert-level AI model responses using detailed rubrics, with clear written rationales.
- Check the reasoning steps in model answers in fields such as mathematics, coding, law, medicine and finance.
- Fact-check complex claims against primary sources and verify their accuracy.
- Assess answers against content policies, style guides and edge cases.
- Review other raters' work, align grading standards, and help develop or refine grading guidelines.
- Assess search relevance and give feedback during live sessions.
What you need
- At least 5 years of professional experience in rating, review, editing, fact-checking, trust and safety, or a closely related domain.
- Native or near-native written English.
- Experience grading or reviewing expert-level work against rubrics, policies or style guides.
- Experience leading or reviewing other raters, such as serving as a QA lead, rubric owner or reviewer of reviewers.
- An advanced degree, professional license, or deep working expertise in the field you grade.
- Hands-on AI data annotation, search-quality rating, or grader-guideline writing experience is a plus.
Open to applicants anywhere in the world.
How to apply
Get the free i'llTip app to see the employer's details and apply. i'llTip's AI can tailor your résumé to this job, and you'll get alerts for similar roles.
Apply on i'llTipPrefer the website? Continue on illtip.comOr get the app: Android · iPhone
🔔 Get new jobs like this first: join Tech & AI Jobs on Telegram · or get personal job alerts
More AI & Machine Learning jobs
AI Agent Evaluation and RL Environment Specialist (Remote - Worldwide)
Pay: $35/hr - $70/hr (hourly rate). Open to applicants who can work remotely from any country. This is a paid talent pool for contractors supporting AI research projects on autonomous software agents and reinforcement le…
AI Response Evaluator, Talent Pool Associate (Remote - Worldwide)
Pay: $20/hr - $45/hr (hourly rate). This is a talent pool for project-based work that helps evaluate and improve AI models. You will review model outputs and give structured feedback so that AI systems are more accurate,…
AI Data Annotation Expert, 5+ Years (Remote - Worldwide)
Pay: $40/hr - $75/hr (hourly rate). Fully remote; no office or country restriction stated. This is an expert pool for an annotation network that supports advanced AI research projects. Contributors are matched to project…
AI Safety Red Team Expert (Remote - Worldwide)
Pay: $75/hr - $175/hr (hourly rate). Fully remote project work with no office requirement. This is an expert talent pool supporting AI safety testing projects for model developers. Members take on evaluation work coverin…
AI Evaluation Associate (Remote - Worldwide)
Pay: $30/hr - $60/hr (hourly rate). This is a remote role with a company that runs human-led evaluations of AI models. You will help measure what models can and cannot do by writing rigorous test questions, building scor…
Related job searches
Remote AI JobsAI jobsContract jobsSenior jobsWork From Home jobs in NigeriaWork From Home jobs