AI Platform Engineer, Training and Inference

About this role

Saviynt is seeking an AI Platform Engineer to manage and optimize distributed training and inference systems for their AI-powered identity platform. This role focuses on ensuring the efficient deployment and scaling of AI models, enhancing operational effectiveness while maintaining security and compliance.

What you'll do

• Manage the Ray ecosystem, including KubeRay on GKE
• Operate distributed training with Ray Train and H100 clusters
• Build and manage the LLM inference mesh using Ray Serve
• Optimize inference performance and autoscaling features
• Design model routing layers for efficient deployment
• Oversee the full model promotion lifecycle and retrain pipelines

What they're looking for

• Experience in ML engineering and MLOps
• Proficiency in Ray Train, Serve, Core, and Data
• Hands-on knowledge with LLM serving engines
• Familiarity with distributed training and RL concepts
• Experience with model lifecycle operations
• Strong programming skills in Python and PyTorch
• Understanding of vector databases and ANN indexing
• Knowledge of quantization techniques (nice to have)

Benefits

• Competitive total rewards package
• Opportunities for learning and career advancement
• Potential for discretionary bonus participation
• [unknown]
• [unknown]
• [unknown]