AI Response Evaluator, Talent Pool Associate (Remote - Worldwide)
Pay: $20/hr - $45/hr (hourly rate). This is a talent pool for project-based work that helps evaluate and improve AI models. You will review model outputs and give structured feedback so that AI systems are more accurate, helpful and safe. Projects usually last from one week to several months, and pay depends on the project, subject field and your experience. Pay is based on hours you actually work.
What you'll do
- Compare two or more model responses and rank them by quality.
- Score answers for accuracy, helpfulness, tone and safety using detailed rubrics.
- Check each step of a model's reasoning and mark it as correct or incorrect, with a short written rationale.
- Fact-check claims against reliable sources and check responses against content and style guidelines.
- Rate search result relevance and give live feedback on model outputs during scheduled sessions.
What you need
- Zero to five years of relevant professional experience.
- Native or near-native written English with strong attention to detail.
- Experience rating, reviewing, fact-checking, editing, moderating or grading work against written guidelines is a plus.
- A degree or work background in a field you can judge, such as writing, journalism, research, teaching, trust and safety, mathematics, science, law, medicine or finance.
- Ability to apply a rubric or style guide consistently across many items.
- Experience with AI data evaluation platforms or search quality or ads rating is useful but not required.
Open to applicants anywhere in the world.
How to apply
Get the free i'llTip app to see the employer's details and apply. i'llTip's AI can tailor your résumé to this job, and you'll get alerts for similar roles.
Apply on i'llTipPrefer the website? Continue on illtip.comOr get the app: Android · iPhone
🔔 Get new jobs like this first: join Tech & AI Jobs on Telegram · or get personal job alerts
More AI & Machine Learning jobs
Senior AI Rater and Reviewer (Remote - Worldwide)
Pay: $50/hr - $120/hr (hourly rate). This is a remote, project-based opportunity with an expert network that trains and evaluates AI systems. Senior raters grade complex model outputs against detailed rubrics and also he…
AI Agent Evaluation and RL Environment Specialist (Remote - Worldwide)
Pay: $35/hr - $70/hr (hourly rate). Open to applicants who can work remotely from any country. This is a paid talent pool for contractors supporting AI research projects on autonomous software agents and reinforcement le…
AI Data Annotation Expert, 5+ Years (Remote - Worldwide)
Pay: $40/hr - $75/hr (hourly rate). Fully remote; no office or country restriction stated. This is an expert pool for an annotation network that supports advanced AI research projects. Contributors are matched to project…
AI Safety Red Team Expert (Remote - Worldwide)
Pay: $75/hr - $175/hr (hourly rate). Fully remote project work with no office requirement. This is an expert talent pool supporting AI safety testing projects for model developers. Members take on evaluation work coverin…
AI Evaluation Associate (Remote - Worldwide)
Pay: $30/hr - $60/hr (hourly rate). This is a remote role with a company that runs human-led evaluations of AI models. You will help measure what models can and cannot do by writing rigorous test questions, building scor…
Related job searches
Remote AI JobsAI jobsEntry-Level jobsContract jobsWork From Home jobs in NigeriaWork From Home jobs