Sr QA Automation Engineer - AI

About Anaconda

Be at the center of AI

With more than 47 million users, Anaconda is the most popular operating system for AI providing access to the foundational open-source Python packages used in modern AI, data science, and machine learning through a seamless platform. We pioneered the use of Python for data science, championed its vibrant community, and continue to steward open-source projects that make tomorrow’s innovations possible. Our enterprise-grade solutions enable corporate, research, and academic institutions around the world to harness the power of open source for competitive advantage, groundbreaking research, and a better world.

What people love most about working here? We’re not just a company, we’re part of a movement. Our dedicated employees and user community are democratizing data science and creating and promoting open-source technologies for a better world, and our commercial offerings make it possible for enterprise users to leverage the most innovative output from open source in a secure, governed way.

Summary:

Anaconda is seeking a talented Senior AI Engineer with QA expertise to join our rapidly growing AI team. In this role, you’ll have the exciting opportunity to work across multiple platforms, focusing on building and testing AI applications and tools, maintaining test frameworks, and ensuring the reliability of our AI core components. This position offers an excellent opportunity to leverage your QA expertise within the AI and machine learning ecosystem while implementing automation solutions that enhance testing efficiency and ensure the quality and reliability of AI models, pipelines, and systems across our offerings.

What You’ll Do:

• Design, develop, and maintain automated test frameworks and test suites for AI/ML-based applications, including APIs, models, and end-to-end workflows
• Design and develop AI-driven​ agents for quality assurance (QA) to boost overall productivity. Our objective is to build intelligent agents capable of generating code and tests, creating QA metrics dashboards, and automatically logging defects from pipeline failures, along with other efficiency-enhancing tasks
• Collaborate with cross-functional teams, including AI/ML engineers, data scientists, and product managers, to define test strategies and ensure high-quality deliverables
• Develop and execute test plans, test cases, and scenarios tailored to AI​ applications, focusing on model accuracy, performance, and reliability
• Perform in-depth testing of machine learning models, including validation, inference testing, and edge-case analysis
• Implement continuous testing processes integrated with CI/CD pipelines to enable rapid and reliable releases
• Integrate responsible AI practices into QA workflows by monitoring for potential biases, fairness concerns, and unintended outcomes
• Advocate for best practices in AI testing and validation by creating and maintaining testing guidelines tailored to AI systems and large language models (LLMs)
• Drive continuous improvement of the QA automation stack, including Web, CLI, and API automation, to support the diverse needs of our team
• Analyze test results, troubleshoot flaky tests, and raise defects in Jira with clear, actionable information
• Stay updated on advancements in AI/ML technologies and apply best practices for testing AI systems
• Mentor junior QA engineers and provide guidance on testing strategies for complex AI systems

What You Need:

• 8+ years of experience in developing robust and reliable automation scripts for applications and APIs, including experience in QA automation, particularly in AI/ML projects and related domains
• Experience in building AI-driven agents and developing custom GPTs to enhance productivity through automation, intelligent testing, and process optimization
• Strong background in Python development and familiarity with AI/ML concepts and testing methodologies
• Experience with testing models for performance, accuracy, reliability, and identifying biases