Staff Machine Learning Engineer

About Us:

Fieldguide is establishing a new state of trust for global commerce and capital markets through automating and streamlining the work of assurance and audit practitioners specifically within cybersecurity, privacy, and ESG (Environmental, Social, Governance). Put simply, we build software for the people who enable trust between businesses.

We’re based in San Francisco, CA, but built as a remote-first company that enables you to do your best work from anywhere. We're backed by top investors including Bessemer Venture Partners, 8VC, Floodgate, Y Combinator, DNX Ventures, Global Founders Capital, Justin Kan, Elad Gil, and more.

We value diversity — in backgrounds and in experiences. We need people from all backgrounds and walks of life to help build the future of audit and advisory. Fieldguide’s team is inclusive, driven, humble and supportive. We are deliberate and self-reflective about the kind of team and culture that we are building, seeking teammates that are not only strong in their own aptitudes but care deeply about supporting each other's growth.

As an early stage start-up employee, you’ll have the opportunity to build out the future of business trust. We make audit practitioners’ lives easier by eliminating up to 50% of their work and giving them better work-life balance. If you share our values and enthusiasm for building a great culture and product, you will find a home at Fieldguide.

About the Role

As a Staff Machine Learning Engineer at Fieldguide, you will lead the development of next-generation AI-driven features on our platform, transforming the audit and advisory industry through cutting-edge generative AI solutions. You’ll focus on applying advanced Machine Learning (ML) and Large Language Models (LLMs) to solve complex problems for our customers, while guiding the technical direction of our ML team in a high-growth startup environment. This role is both strategic and hands-on – you will set best practices for our generative AI efforts and also dive into coding and architecture as needed to drive critical projects from concept to production.

In this role, you will be the go-to expert for generative AI at Fieldguide. You’ll establish standards for prompt engineering, context management, and model evaluation, ensuring our use of LLMs is effective, safe, and scalable. As a Staff MLE, you will also act as a multiplier for the entire engineering team: reviewing architectures for AI features, mentoring other engineers, and fostering a culture of excellence in ML. You’ll collaborate closely with cross-functional stakeholders – from product managers and designers to even high-profile clients – to translate business needs into technical solutions and to communicate how our AI-driven approach creates value. This is a unique opportunity to shape the future of Fieldguide’s AI capabilities and establish yourself as a technical leader in the burgeoning field of generative AI.

What You’ll Do

• Architect Generative AI Solutions: Design and oversee the architecture of systems that leverage LLMs and retrieval-augmented generation (RAG) techniques. You will make key decisions on how we integrate LLMs with our existing platform and data stores, including building agent-based frameworks where LLMs interact with tools and knowledge bases (e.g. creating AI “co-pilots” for auditors). You’ll conduct rigorous architectural reviews and ensure our designs meet high standards for scalability, security, and reliability.

• Establish Prompt Engineering Best Practices: Develop and codify best practices for prompt engineering and context management in our AI applications. You will guide the team in crafting effective prompts, choosing model parameters, and managing conversation context to optimize LLM performance. This includes building internal libraries or templates for prompts and educating engineers on how to avoid common failure modes. By setting this technical quality bar, you’ll ensure consistency and excellence in how we build GenAI features.

• Develop Evaluation Frameworks: Create and implement frameworks to evaluate generative AI outputs for quality, accuracy, bias, and safety. You will define ML performance metrics specific to generative models (e.g. factual correctness rates, relevance scores, user feedback loops) and possibly leverage tools or develop custom evaluators (such as automated prompts or human-in-the-loop reviews). These evaluation strategies will inform model improvements and help establish standards for GenAI system evaluation across the company.

• Lea