AI Engineer - Harness & Evals

About this role

Build is seeking an AI engineer to develop core infrastructure for agentic AI systems serving real estate and built-world enterprises. You'll own the agent runtime, evaluation systems, retrieval layers, and observability platform that enable reliable, scalable AI workflows in high-stakes production environments.

What you'll do

• Design and build agent platform infrastructure including runtime, tool orchestration, workflow state management, and resumability
• Create comprehensive evaluation systems to measure agent behavior, groundedness, workflow completion, and detect regressions
• Develop context assembly and retrieval systems for documents, structured data, and project state
• Build observability and tracing systems for production quality metrics, model versions, and failure analysis
• Improve LLM reliability through structured outputs, validation layers, guardrails, and failure recovery mechanisms
• Partner with product engineers to establish reusable primitives, SDKs, and platform capabilities

What they're looking for

• Backend/systems engineering for production AI platforms
• LLM frameworks and agent architecture
• Retrieval systems and context management
• Observability, tracing, and monitoring systems
• Python or similar systems programming language
• Distributed systems and workflow orchestration
• Evaluation and testing frameworks for AI systems
• Security and reliability engineering