Research Engineer, Benchmarks (Visa Sponsorship - Singapore)

Visa sponsorship available. A small, technical team focused on building rigorous evaluations for advanced AI agents. You will own the design and build of domain-specific benchmarks that help research teams and customers understand how agents perform on realistic workflows. The role is hands-on and suits someone who is comfortable in an early-stage environment.

What you'll do

• Design, implement and maintain internal benchmarks that test advanced agents on domain-specific tasks.
• Work with subject-matter experts to turn real-world workflows into benchmark tasks and evaluation criteria.
• Build reliable infrastructure to run models and agents against benchmark tasks at scale.
• Develop metrics and analyses that measure benchmark difficulty, reliability and failure modes.
• Check whether benchmark results match real-world outcomes and user needs.
• Write clear technical documentation and reports on benchmark methods and results.

What you need

• 2 to 4 years of experience in software, machine learning or research engineering, including at least two years building AI benchmarks, evaluation infrastructure or agent environments.
• Technical education or strong equivalent fundamentals.
• Proficiency with Python, Docker and Linux.
• Hands-on experience designing or operating evaluations or environments for AI agents or large language models.
• Experience with agent or model execution infrastructure, evaluation metrics or statistical analysis, and benchmark validation.
• Strong technical writing and attention to detail.

How to apply

Get the free i'llTip app to see the employer's details and apply. i'llTip's AI can tailor your résumé to this job, and you'll get alerts for similar roles.

Prefer the website? Continue on illtip.comOr get the app: Android · iPhone

🔔 Get new jobs like this first: join Tech & AI Jobs on Telegram · or get personal job alerts