Research Engineer, Data & Evals (Remote - Worldwide)

This is a research engineering role at a fast-growing AI company building agentic systems for legal and professional services work. The team recently post-trained its first model for legal tasks and is now expanding its post-training effort. The role focuses on the data and measurement infrastructure behind training environments, benchmarks, and evaluations, where correctness is hard to verify automatically.

What you'll do

• Build synthetic data pipelines that generate, filter, and curate training and evaluation samples for complex legal reasoning and agentic workflows.
• Design evaluations and verifiers that reflect real legal work, calibrate them against expert judgment, and investigate where they fail.
• Build and maintain capability benchmarks and product-focused evals that track model performance on real customer workflows.
• Work with legal domain experts to define quality metrics, rubrics, and grading mechanisms.
• Own training and evaluation data quality, including provenance, coverage, and safeguards against contamination.
• Partner with researchers to investigate model failures and separate real gains from misleading metrics.

What you need

• Experience building data pipelines, ML evaluation systems, or experimentation tooling, with strong Python skills and production-quality code.
• A rigorous approach to measurement, including statistical reasoning, experimental design, and an understanding of when metrics mislead.
• Ability to turn ambiguous goals or expert judgment into structured datasets, rubrics, and measurable criteria.
• Experience with synthetic data, LLM evaluations, automated raters, or preference data is a plus; strong data and measurement engineering close to ML development is valued even without prior LLM work.
• Comfort working closely with researchers and legal experts in a small, high-ownership team.

Open to applicants anywhere in the world.

How to apply

Get the free i'llTip app to see the employer's details and apply. i'llTip's AI can tailor your résumé to this job, and you'll get alerts for similar roles.

Prefer the website? Continue on illtip.comOr get the app: Android · iPhone

🔔 Get new jobs like this first: join Tech & AI Jobs on Telegram · or get personal job alerts