Software Engineer AI Gateway

About this role

Join Kong's AI Gateway engineering team to build advanced API management software for AI and LLM applications. You'll develop core features like multi-LLM routing, semantic caching, and load balancing while working on a distributed, scalable platform trusted by Fortune 500 companies.

What you'll do

• Build and maintain API management software for AI/LLMs and its underlying infrastructure
• Implement innovative features including proxying, load balancing, and database support
• Troubleshoot bugs, optimize performance, and execute low-level system optimizations
• Integrate Kong with third-party deployment platforms and tools
• Design highly scalable, distributed, and resilient applications

What they're looking for

• Golang or Rust programming
• Server-side application development (3+ years)
• Linux systems troubleshooting and profiling
• Distributed systems and networking stack knowledge
• L4/L7 protocol and transport expertise
• AI/LLM concepts and applications development
• NGINX/OpenResty/Kong/LuaJIT (bonus)
• Envoy or Istio experience (bonus)

Benefits

• Remote work with distributed team
• Work on cutting-edge AI gateway technology
• Contribute to infrastructure powering Fortune 500 companies
• Strong ownership and technical autonomy
• Documentation ownership for your products