Anthropic
Performance Engineer, Inference Engine
About this role
Anthropic seeks a Performance Engineer to optimize the inference engine that serves Claude at scale, handling token batching, memory management, and distributed computation across accelerator platforms. You'll work on throughput, latency, and reliability across hardware and cloud platforms using deep systems knowledge and performance profiling.
What you'll do
- Optimize inference engine performance across throughput, cost, latency, and reliability on all accelerator and cloud platforms
- Manage token path execution including request batching, model layout across chips, memory coordination, and forward pass orchestration
- Profile systems and model performance bottlenecks using hardware metrics (FLOPs, HBM, PCIe, RDMA, network links)
- Build observability and instrumentation to identify performance gaps and measure the impact of optimizations
- Collaborate with safety and safeguards teams to ensure production safety systems maintain efficiency without compromising robustness
- Maintain model quality and consistency across platforms and over time through infrastructure improvements
What they're looking for
- Systems programming (Rust, C++, or similar)
- LLM inference architecture (prefill/decode, compute/memory/interconnect tradeoffs)
- Hardware performance analysis and benchmarking
- GPU/Accelerator programming
- Distributed systems design and coordination
- Performance profiling and observability tools
- Transformer architecture understanding
- Code quality and testing practices
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
Anthropic
Anthropic builds Claude, an AI assistant, and is hiring for engineering roles across infrastructure, data systems, and security that support both AI research operations and the company's internal technology needs. The company seeks infrastructure engineers, systems integrators, data scientists, and security specialists to build production-scale systems for training data pipelines, financial operations, developer productivity measurement, research infrastructure, and server firmware security.
- Website
- anthropic.com
Likely interview questions
- Walk us through your mental model of how prefill and decode phases map onto accelerator resources—compute, memory bandwidth, and interconnect—and what the host CPU does during each phase.
- Describe a time you identified and resolved a critical performance bottleneck in a complex system. How did you measure, model, and validate your fix?