RL Environment and Agent Evaluation Specialist (Remote - Worldwide)
Pay: $35/hr - $70/hr (hourly rate). Open to applicants in any country; project-based work with flexible hours. This is a paid talent pool for an AI company working on AI agents and reinforcement learning. Contributors help build practice environments and assess how agents behave when they interact with software and business workflows. Pay depends on the project, subject area and your experience, and is based on hours worked. Projects typically run from about one week to several months.
What you'll do
- Build reinforcement learning environments in Python and Docker where agents can take actions and receive feedback.
- Convert business processes such as ERP steps, CRM updates and spreadsheet operations into trainable task scenarios.
- Write tasks, connect tools and REST APIs, and build checks that verify whether an agent's actions were correct.
- Create and grade agent demonstrations, including screen recordings of task execution.
- Collect execution traces, evaluate agent trajectories and investigate why failures happen.
What you need
- Zero to five years of experience in software engineering, QA automation, machine learning or RL engineering, operations analysis, or heavy use of business software.
- Strong Python skills, or experience building automations and reports in Excel, Salesforce or an ERP system.
- Hands-on experience testing, scripting or automating multi-step software tasks such as test cases, API calls or UI steps.
- Useful extras: Docker, REST APIs, Gymnasium/OpenAI Gym-style environments, pytest, Playwright or Selenium.
- Solid software fundamentals or practical knowledge of business workflows. Prior AI data work is not required.
Open to applicants anywhere in the world.
How to apply
Get the free i'llTip app to see the employer's details and apply. i'llTip's AI can tailor your résumé to this job, and you'll get alerts for similar roles.
Apply on i'llTipPrefer the website? Continue on illtip.comOr get the app: Android · iPhone
🔔 Get new jobs like this first: join Tech & AI Jobs on Telegram · or get personal job alerts
More AI & Machine Learning jobs
AI Safety Testing Associate (Remote - Worldwide)
The posting does not state country restrictions. This is a safety testing role with an expert network that supports AI model evaluation projects. You will look for ways AI systems can fail and document your findings so m…
AI Response Evaluation and Feedback Associate (Remote - Worldwide)
Fully remote, no country restriction stated. Paid for hours worked. A project-based expert network that supports the evaluation and improvement of AI models. Contributors give structured feedback that helps model respons…
AI Data Annotation Specialist (Remote - Worldwide)
Pay: $40/hr - $75/hr (hourly rate). This is an expert-level data annotation role supporting advanced AI research teams. You will work on complex, multimodal datasets, take on the hardest and most ambiguous cases, and hel…
AI Evaluation Specialist, Contract Talent Pool (Remote - Worldwide)
Pay: $30/hr - $60/hr (hourly rate). This is a contract position in a specialist talent pool that supports human-data projects evaluating AI models. Contributors design and run assessments that measure what models can and…
Senior Software Engineer I/II, Operational AI (Remote - US and Canada)
This is a senior engineering role on the Operational AI team at a public company that builds connected operations software for industries such as transportation, construction, agriculture and field services. The team tur…
Related job searches
Contract jobsWork From Home jobs in NigeriaWork From Home jobs