agencywithin
AI Engineer
Long Island City, New YorkmidAdded 2 days ago
About this role
Join our team to architect and develop an internal AI agent platform that powers productivity across the organization. You'll design LLM-integrated applications with tool orchestration, context management, and evaluation frameworks while collaborating with cross-functional teams to expand the platform into document and workflow automation.
What you'll do
- Design and build AI applications leveraging LLM APIs, tool calling, and structured outputs
- Develop agent architectures integrating SQL, Python execution, and marketing platform APIs
- Implement reliability safeguards, retry logic, and hallucination mitigation strategies
- Build evaluation frameworks for LLM systems including multi-turn and regression testing
- Expand platform capabilities into document generation and Google Workspace integrations
- Collaborate with ML engineers, product managers, and full-stack engineers on system reliability
What they're looking for
- Python (production systems)
- LLM APIs (OpenAI, Anthropic, Vertex AI)
- Prompt engineering and multi-turn conversation design
- Tool/function calling and structured outputs
- SQL and database integration
- Cloud platforms (GCP, AWS, Azure)
- API integration and system debugging
- RAG architectures and vector search (plus)
Benefits
- Unlimited vacation policy
- Comprehensive medical, dental, and vision insurance
- 401(k) plan with matching
- Hybrid work opportunity
- Professional development program
- Dog-friendly office and phone stipend
Opens the official application on the employer’s site. No login required.