xAI
Software Engineer - Evals
About this role
Build and maintain the evaluation platform that measures AI model capabilities and behaviors at xAI. You'll design assessment systems, manage execution infrastructure, and create datasets that drive frontier AI research in a fast-paced, mission-driven environment.
What you'll do
- Design and build evaluation instruments to measure model capabilities, behaviors, and safety
- Create and manage datasets with grading schemes for model assessment and feedback
- Develop and maintain execution infrastructure for running evaluations at scale
- Diagnose model failures and propose both short-term and long-term improvements
- Ship high-quality systems that engineers across the organization rely on daily
- Translate qualitative notions of quality into measurable, concrete evaluation criteria
What they're looking for
- Software engineering and production code writing (3+ years)
- Product design and systems thinking
- Attention to detail and quality assurance
- First principles reasoning
- AI/ML workflows and model evaluation (bonus)
- Data annotation tools and processes (bonus)
- Internal tools or design system development (bonus)
- Infrastructure and execution management
Benefits
- Equity compensation
- Comprehensive medical, vision, and dental coverage
- 401(k) retirement plan
- Short and long-term disability insurance
- Life insurance
- Various discounts and perks
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
xAI
xAI builds advanced AI infrastructure and systems, including the Grok model inference platform and Colossus GPU cluster. The company is hiring Mechanical, Electrical, and Facilities Engineers to design and maintain its data center operations, as well as Software Engineers to optimize high-performance inference systems and datacenter networking.
View all jobs at xAILikely interview questions
- Walk us through a high-quality project you shipped—what made it exceptional and how did you ensure attention to detail?
- Describe your experience translating fuzzy product requirements into concrete, measurable success criteria.