Anyscale
Software Engineer, Infrastructure
About this role
Anyscale is seeking an Infrastructure Software Engineer to build the control and data plane systems that power its Ray-based distributed AI platform. You'll design scalable orchestration services, optimize scheduling for heterogeneous clusters, and ensure reliable execution of ML workloads across cloud and on-premises environments.
What you'll do
- Design and scale services that orchestrate Ray clusters across cloud and on-prem environments
- Optimize control plane components for large-scale distributed AI/ML workloads
- Build intelligent scheduling and resource management systems for heterogeneous compute clusters
- Develop features for reliability, performance, scalability, and observability of managed Ray workloads
- Support accelerator integration (GPUs, TPUs) and container image management
- Provide on-call support and collaborate with customer teams on infrastructure troubleshooting
What they're looking for
- Kubernetes and container orchestration
- Cloud platforms (AWS, Azure, GCP)
- Distributed systems design and implementation
- Control plane and data plane development
- High-performance systems optimization
- Production code quality and testing
- Scheduling and resource management algorithms
- On-call support and debugging
Benefits
- Work on open-source Ray and proprietary Anyscale products
- Collaborate with distributed systems and ML experts
- Competitive funding from Andreessen Horowitz, NEA, and Addition
- Located in San Francisco or Palo Alto, CA
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
Anyscale
Anyscale builds Ray, an open-source distributed computing framework and enterprise platform for scaling AI workloads across Kubernetes and cloud providers. The company is hiring forward-deployed engineers to work embedded with customers, software engineers to develop Ray Core, LLM inference specialists, and customer support engineers who combine technical expertise with post-sale success.
- Website
- anyscale.com
Likely interview questions
- Describe your experience building or maintaining distributed systems at scale—what were the key challenges and how did you address them?
- How have you optimized control plane or orchestration systems for performance, and what metrics did you use to measure success?