Founding Engineer, Environments and Data (Visa Sponsorship - San Francisco, United States)

Visa sponsorship available. A fast-growing AI-focused company is hiring a founding engineer to build evaluation environments and benchmarks that measure whether AI agents can complete real company work at or above human level. This is a founding role in a small team, with a strong emphasis on research and engineering rather than infrastructure alone. The work shapes how agent competence is measured and whether agents are ready for production use.

What you'll do

• Design synthetic company environments with realistic email, records, and files that support extended agent use.
• Build robust graders and evaluation systems that can judge agent output even where human reviewers are not expert enough to do so.
• Develop benchmarks that correlate with real-world production performance.
• Identify and fix evaluation failures, including reward hacking and benchmark gaming.
• Simulate software tools such as inboxes and CRMs with enough fidelity to support agent interactions at scale.

What you need

• Hands-on experience with synthetic data pipelines, benchmarks, evaluations, or post-training.
• Experience building environments or benchmarks used to train or evaluate agents, and designing graders or evaluation systems for AI agents.
• Experience simulating complex software systems for agent tool use, and finding benchmark flaws or fixing reward hacking.
• At least 1 to 2 years at a fast-growing or AI-focused startup, with relevant AI research or engineering experience.
• Strong startup experience (such as founding a company or joining an accelerator) or high-level competition experience in areas such as mathematics, programming, debate, or games.
• A clear belief that AI agents can outperform humans at many computer-based tasks.

How to apply

Get the free i'llTip app to see the employer's details and apply. i'llTip's AI can tailor your résumé to this job, and you'll get alerts for similar roles.

Prefer the website? Continue on illtip.comOr get the app: Android · iPhone

🔔 Get new jobs like this first: join Tech & AI Jobs on Telegram · or get personal job alerts