Software Engineer, Bulk/Interactive Inference

About this role

Join Waymo's ML Platform team to design and optimize the inference infrastructure powering autonomous driving AI systems. You'll build scalable, high-performance pipelines for model hosting and inference across diverse use cases, from LLMs to dataset generation and auto-labeling.

What you'll do

• Develop and scale Waymo's inference platform for high throughput and low latency performance
• Partner with cross-functional teams to host and integrate internal and external ML models, including large language models
• Optimize inference efficiency on large models to maximize throughput and reduce operational costs
• Deploy model inference solutions across multiple applications including distillation, evaluation, and active learning
• Build and maintain critical data and ML pipelines supporting developer velocity
• Collaborate with ML modeling teams on infrastructure solutions

What they're looking for

• C++ programming
• Distributed systems design and implementation
• ML model inference and optimization
• Scalable infrastructure development
• Data pipeline design
• Performance optimization and tuning
• System architecture and design
• Large-scale data handling

Benefits

• Discretionary annual bonus program
• Equity incentive plan
• Generous company benefits package
• Hybrid work arrangement
• Impact on autonomous driving technology
• Work with cutting-edge ML systems

Likely interview questions

• Describe your experience building or optimizing distributed systems at scale. What challenges did you face?
• Tell us about a time you optimized inference latency or throughput for ML models. What techniques did you use?