Lead Product Manager

Lead Product Manager – AI Search & Asset Intelligence
 

Job Title: Lead Product Manager
Reporting To: Vice President – AI Product
 

THE OPPORTUNITY

Are you a technical product manager with a passion for how AI can solve the "needle in a haystack" problem for enterprise data?

Tricentis is the industry’s #1 Continuous Testing platform. Our customers manage thousands of test assets, yet they often face a critical challenge: Discoverability. When users cannot find existing assets, they recreate them—leading to redundancy, maintenance debt, and inefficiency.

We are looking for a Lead Product Manager to own our AI Search and Asset Intelligence strategy. You will leverage RAG (Retrieval-Augmented Generation), Vector Search, and Recommender Systems to transform how users find, reuse, and optimize their testing portfolios.

WHAT YOU WILL BE DOING

• Own the "Asset Intelligence" Roadmap: You will drive the strategy for AI-enabled asset discovery, focusing on reducing redundancy and increasing the re-use of testing components across the Tricentis portfolio.

• Build Technical AI Products: You will define the requirements for our Search and RAG architecture, making high-stakes decisions on indexing strategies, relevance ranking, and context windows.

• Bridge the Gap: You will act as the translator between Data Science/AI engineering teams and business stakeholders, converting complex technical capabilities into tangible customer value.

• Drive Execution: Unlike a purely strategic role, this is a hands-on Lead IC role. You will write detailed technical specs, groom backlogs with engineering, and measure model performance (precision/recall) against business metrics (user retention/asset reuse rates).

RESPONSIBILITIES

• Define Agentic Success Metrics: Move beyond vanity metrics like Click-Through Rate (CTR). You will define and track Task Success Rate, Goal Completion, Steps-to-Solution, and Recovery Rate to measure how effectively agents solve user problems without human intervention.

• Manage Agent "Skills" & Tooling: Define the "tools" (APIs, functions, and data sources) your agents can access. You will specify the input/output contracts that allow the AI to interact with other Tricentis products (e.g., "Open JIRA Ticket," "Scan Repository," "Execute Test").

• Orchestrate Multi-Turn Reasoning: Design experiences where agents maintain Short-Term Memory (context of the current session) and Long-Term Memory (past user preferences), ensuring the system doesn't lose context during complex, multi-step workflows.

• Evaluation & Ground Truth: Establish "Golden Datasets" and evaluation pipelines to test for Hallucination Rate and Reasoning Accuracy before deployment. You will be responsible for the trade-offs between model latency and reasoning depth.

• Cross-Portfolio Integration: Work across multiple Tricentis product lines to ensure a unified search experience—allowing a user in one tool to seamlessly find and import assets from another.

TECHNICAL KNOWLEDGE

• Agentic Frameworks: Deep understanding of agent architectures like ReAct (Reason + Act) and Chain-of-Thought (CoT) reasoning. You should understand how agents decompose high-level goals into sub-tasks.

• Enterprise Data Privacy & Security:

• • RBAC for RAG: Knowledge of implementing Role-Based Access Control at the vector/chunk level to ensure users never retrieve data they aren't authorized to see.

  • Data Minimization: Experience designing pipelines that redact PII (Personally Identifiable Information) and sensitive secrets before data enters the vector store or context window.

  • Zero-Trust Retrieval: Understanding of ensuring that every tool call or retrieval step is verified against the user’s permissions token.

• Vector Database & RAG Strategy: Familiarity with indexing strategies (sparse vs. dense vectors), chunking methods, and semantic reranking to improve retrieval relevance.

• LLM Evaluation: Ability to design "LLM-as-a-Judge" frameworks to automatically grade agent outputs against defined rubrics.
   

WHAT YOU NEED

Basic Qualifications (Must Haves)

• 5-8+ Years of Product Management experience, with at least 2+ years dedicated to Technical Product Management or AI/Data products.

• AI/ML Fluency: Demonstrated experience shipping products powered