Clera
Research Engineer, Benchmarks
About this role
Join a San Francisco-based startup building AI evaluation benchmarks as a Research Engineer. You'll design and implement rigorous benchmarks for frontier AI agents, partner with domain experts to create realistic workflows, and develop infrastructure to measure real-world agent performance at scale.
What you'll do
- Design, implement, and maintain high-quality benchmarks for evaluating frontier AI agents on domain-specific tasks
- Partner with subject-matter experts to define realistic workflows and translate them into benchmark evaluation criteria
- Build scalable infrastructure to reliably run AI models and agents against benchmark tasks
- Develop metrics and statistical analyses to assess benchmark difficulty, reliability, and failure modes
- Validate benchmark performance against real-world evaluations and customer needs
- Write clear documentation and technical reports communicating benchmark results to research and engineering audiences
What they're looking for
- Python, Docker, and Linux infrastructure development
- AI benchmark design and evaluation framework implementation
- Large language model and AI agent evaluation experience
- Statistical analysis and metrics development
- Reinforcement learning pipelines and agent evaluation
- Workflow analysis across technical and business domains
- Technical writing and communication
- Infrastructure scaling and reliability engineering
Benefits
- Salary: $150,000–$250,000 USD annually
- Early-stage equity participation
- Visa sponsorship available
- On-site team environment in San Francisco with highly technical researchers
- High-ownership role at intersection of research and engineering
- Opportunity to work on critical infrastructure for frontier AI evaluation
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
Clera
Clera builds an agentic operating system that automates complex workflows and processes through AI agents, with a platform designed to simplify distributed infrastructure management for developers. The company is hiring Founding Engineers, Customer Engineers, and Product Engineers to develop both backend systems and user-facing interfaces across their AI automation products.
View all jobs at CleraLikely interview questions
- Describe a benchmark or evaluation infrastructure you've built—what made it reliable and how did you validate its real-world relevance?
- How have you approached designing metrics to assess benchmark difficulty and reliability, and what statistical analyses did you use?