Artificial Analysis
Forward Deployed Engineer - Media Generation
About this role
Artificial Analysis, a leading AI benchmarking company, is seeking a Solutions Engineer to manage their media generation benchmarking pipeline. You'll oversee image and video generation evaluations, coordinate human preference studies, and serve as a technical liaison with model providers, ensuring rigorous quality control across all benchmarking operations.
What you'll do
- Execute image and video generation evaluations across models following standardized protocols
- Design and manage human preference evaluation studies including participant coordination and quality assurance
- Process preference voting data and produce benchmark results and analysis
- Own the complete evaluation pipeline from prompt execution through published results
- Act as technical point of contact for media generation model providers regarding results and methodology
- Monitor data quality, detect anomalies, and maintain consistency across evaluation rounds
What they're looking for
- Python scripting and API integration
- Technical operations or data operations experience
- Research study management and data collection pipeline design
- Process management and workflow reliability
- Written and verbal communication
- Experience with crowdsourcing or human evaluation platforms
- Familiarity with generative AI models (Midjourney, DALL-E, Stable Diffusion, etc.)
- Data analysis or research operations background
Benefits
- Competitive compensation with equity
- Shape AI development priorities at leading labs
- Exposure to cutting-edge models and capabilities across the AI industry
- Work with major AI labs and enterprises as trusted partner
- Join a fast-growing 35+ person team backed by prominent AI investors
- Flexible location across San Francisco, Sydney, or Melbourne
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
Artificial Analysis
Artificial Analysis builds AI benchmarking infrastructure that evaluates and compares language models and media generation systems. The company is hiring Solutions Engineers to operate benchmarking pipelines, manage evaluations, and serve as technical liaisons between its infrastructure and AI lab customers.
View all jobs at Artificial AnalysisLikely interview questions
- Walk us through your experience managing end-to-end data pipelines or research studies. How did you ensure data quality and consistency?
- Tell us about a time you worked with Python to automate a repetitive process or manage data collection. What was the scope and outcome?