Software Engineer, AI Agent & LLM

<div> <h4>The Opportunity</h4> <p>We are looking for a Senior AI Agent &amp; LLM Engineer who combines strong software engineering capabilities with a deep focus on AI quality.&nbsp;You will help build and improve the AI systems behind Otter’s conversational knowledge engine and AI Chat, spanning user-facing agent experiences, evaluation systems, and the shared platform required to operate them reliably at scale.</p> <p>This is a hands-on role for someone who can move from an ambiguous product problem to a working production solution, define how quality should be measured, and drive improvements across services, models, and user experience.</p> <h4>Your Impact</h4> <ul> <li>Build AI-agent experiences for Otter AI Chat that help users reason over conversations, retrieve knowledge, and complete complex tasks.</li> <li>Advance Otter’s conversational knowledge engine through better knowledge extraction, annotation, and indexing.</li> <li>Develop robust evaluation datasets, automated graders, regression tests, and release gates for AI quality.</li> <li>Diagnose failures across models, prompts, retrieval, tools, data pipelines, backend services, and product workflows.</li> <li>Build shared agent infrastructure for orchestration, tracing, debugging, retries, sandboxed execution, and observability.</li> <li>Improve task completion, correctness, groundedness, reliability, latency, and cost.</li> <li>Turn production traces, customer feedback, and usage signals into measurable product and model improvements.</li> <li>Collaborate across AI, product, infrastructure, data, security, and application teams to deliver end-to-end capabilities.</li> </ul> <h4>We're looking for someone who</h4> <ul> <li>Has 3+ years of AI Agent engineering, machine learning engineering, or related experience.</li> <li>Brings strong backend or distributed-systems engineering skills.</li> <li>Has hands-on experience building and shipping AI-agent or LLM-powered products.</li> <li>Has a strong focus on AI quality and experience evaluating nondeterministic systems.</li> <li>Can use data, traces, logs, and qualitative examples to identify and resolve complex failures.</li> <li>Works effectively across services, technical domains, and organizational boundaries.</li> <li>Combines strong product judgment with rigorous engineering and evaluation practices.</li> <li>Operates with high agency, strong ownership, and a bias toward action.</li> <li>Can take ambiguous problems from initial exploration through production launch and continuous improvement.</li> <li>Demonstrated ability to use coding agents effectively while rigorously reviewing and controlling the quality of their output is a plus.&