Solutions Engineer — Language Models

About this role

Artificial Analysis seeks a Solutions Engineer to operate and maintain its language model benchmarking pipeline, serving as the technical liaison between the company's infrastructure and AI lab customers. You'll manage model evaluations, debug issues across the stack, and communicate benchmarking results to leading AI organizations.

What you'll do

• Operate Python-based benchmarking pipeline: onboard models, configure evaluations, execute runs, and validate results
• Debug and resolve issues across APIs, data pipelines, and infrastructure; ensure accuracy of published benchmarks
• Serve as primary technical contact for AI lab customers via Slack and video calls to explain results and troubleshoot
• Monitor benchmark runs for anomalies and investigate discrepancies
• Maintain documentation of processes, configurations, and known issues
• Collaborate with engineering team on pipeline improvements and stay current with model ecosystem developments

What they're looking for

• 5+ years client-facing technical experience (solutions/support engineering)
• Strong Python proficiency and debugging across complex systems
• Hands-on experience with AI/ML model APIs (OpenAI, Anthropic, Google, Meta, etc.)
• Excellent written and verbal communication skills
• Attention to detail and ability to work reliably under pressure
• Experience with AI evaluation or benchmarking methodologies (preferred)
• Familiarity with LLM inference infrastructure and metrics (preferred)
• B2B SaaS or developer tools background (preferred)

Benefits

• Remote position in the United States
• Direct influence on how frontier AI is developed across leading labs
• Exposure to cutting-edge AI models and capabilities as they're released
• Relationships with teams at major AI organizations (OpenAI, Google, Anthropic, Meta, etc.)
• Company rapidly scaling (35+ team growing to 100+)
• Backed by prominent industry leaders and investors