Software Engineer, AI Infrastructure

About this role

Join Fireworks AI's infrastructure team to design and build the core systems powering their industry-leading generative AI platform. You'll develop scalable backend infrastructure, optimize performance and reliability, and collaborate with cross-functional teams to deliver a world-class AI service.

What you'll do

• Design and develop scalable backend infrastructure for distributed training, inference, and data pipelines
• Build and maintain core services including LLM CI/CD pipelines, control plane, and model serving systems
• Optimize performance, cost efficiency, and reliability across compute, storage, and networking
• Develop frameworks and safeguards to ensure industry-leading model quality
• Translate research and product requirements into infrastructure solutions with performance and training teams
• Participate in code reviews, technical discussions, and deployment processes

What they're looking for

• Python, Go, or similar programming languages
• ML infrastructure tools (PyTorch, MLflow, Vertex AI, SageMaker, Kubernetes)
• LLM fundamentals (context length, KV cache, prefill optimization)
• Distributed systems design
• Backend services architecture
• Open source inference engines (vLLM, Sglang, TRT-LLM)
• CI/CD pipeline development
• Large-scale systems optimization

Benefits

• Meaningful equity in a Series C startup ($4B valuation)
• Competitive base salary ($175,000–$220,000 USD)
• Comprehensive benefits package
• Work on cutting-edge AI infrastructure challenges
• Collaborate with world-class engineers from Meta and Google
• High ownership and direct impact on product direction