Fireworks AI
Software Engineer, LLM Infrastructure
About this role
Join Fireworks AI's infrastructure team to design and build the core systems powering their industry-leading generative AI platform. You'll develop scalable backend infrastructure, optimize performance and reliability, and collaborate with cross-functional teams to deliver a world-class AI service.
What you'll do
- Design and develop scalable backend infrastructure for distributed training, inference, and data pipelines
- Build and maintain core services including LLM CI/CD pipelines, control plane, and model serving systems
- Optimize performance, cost efficiency, and reliability across compute, storage, and networking
- Develop frameworks and safeguards to ensure industry-leading model quality
- Translate research and product requirements into infrastructure solutions with performance and training teams
- Participate in code reviews, technical discussions, and deployment processes
What they're looking for
- Python, Go, or similar programming languages
- ML infrastructure tools (PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes)
- LLM fundamentals (context length, KV cache, prefill optimization)
- Distributed systems design
- Backend services architecture
- Open source inference engines (vLLM, Sglang, TRT-LLM)
- CI/CD pipeline development
- Large-scale systems optimization
Benefits
- Meaningful equity in a Series C startup ($4B valuation)
- Competitive base salary ($175,000–$220,000 USD)
- Comprehensive benefits package
- Work on cutting-edge AI infrastructure challenges
- Collaborate with world-class engineers from Meta and Google
- High ownership and direct impact on product direction
Opens the official application on the employer’s site. No login required.
Fireworks AI
Fireworks AI builds a generative AI infrastructure platform that enables efficient training and deployment of large language models at scale. The company is hiring research engineers, infrastructure engineers, security engineers, applied ML engineers, and field engineers to develop core systems, optimize performance, secure platforms, and drive customer adoption across enterprise partnerships.
- Website
- fireworks.ai
Likely interview questions
- Describe your experience with ML infrastructure tooling like PyTorch, Kubernetes, or similar platforms. How have you used these in a production setting?
- Walk us through a time you optimized performance or cost efficiency in a distributed system. What were the bottlenecks and how did you address them?