Judgmentlabs
Product Engineer, Agent
- Confirmed live in the last 24 hours
- No salary listed
- Mid level
- Full-time
- On-site · San Francisco
- Added 1 month ago
About this role
This Product Engineer role focuses on building the "Judgment" platform, which helps AI agents learn and improve from real-world production data. You'll own the entire product lifecycle, from customer interaction and problem definition to development and iteration. The position involves designing and building tools to analyze agent behavior, verify changes, and streamline the improvement loop.
What you'll do
- Develop the Judgment Agent for large-scale investigations.
- Build a platform for verifying agent changes through simulations and trajectory replay.
- Design interfaces for engineers to understand agent behavior and debug reasoning loops.
- Create a user experience for monitoring and interacting with investigations.
- Build workflows to convert production data into datasets and regression checks.
- Develop platform infrastructure (workspaces, roles, permissions).
What they're looking for
- Production Systems
- Problem-Solving
- LLMs/Agents
- UI Development
- Communication
- SDK Development
- Data Layer
Opens the application — the Jobs AI extension fills it for you. Set up autofill
Opens the official application on the employer’s site. No login required.
Judgmentlabs
- Industry
- Technology & Software
Likely interview questions
- Describe your experience building and scaling production systems.
- How would you approach debugging a complex reasoning loop in an AI agent?