Staff Quality Engineer - Machine Learning

We're Celonis, the global leader in Process Mining technology and one of the world's fastest-growing SaaS firms. We believe there is a massive opportunity to unlock productivity by placing data and intelligence at the core of business processes - and for that, we need you to join us.

The Role:

As a Staff Quality Engineer, you have a keen understanding of what it takes to deliver a high-quality product as well as strengths in the area of automation coding and delivery. You are responsible for the overall effort to ensure that all innovations from our development team will be implemented in the right way in our software, including the creation and monitoring of automated tests. You are not bothered by getting your hands dirty with manual testing, but you are bothered enough by repetitive tasks to think creatively about how to not have to repeat them again. You believe in doing the job well but you also understand the balance between elegant solutions, time to market, and product quality. In this role, you support our core developers by ensuring a high quality of our Celonis EMS. On the other side, you are the link between development and product management. You have a need for quality assurance perfectionism, and it drives your passion for technology. 

The work you’ll do:

• Operate independently as a member of a cross-functional team, consistently advocating to push quality left and drive customer satisfaction of our product offerings
• Own responsibility for one or more complex areas of testing
• Drive robust and high-quality test coverage with minimal overlap in the team, ensuring the definition of done includes minimal but sufficient automation and recognize the level of automation necessary to reach the goals
• Research, recommend and implement tools and processes as needed with the goal of increasing automation and test coverage
• Collaborate across the EPE Chapter, mentoring other team members and contributing to efforts to improve overall process and quality
• Live by data not opinions, ensuring your team consistently meets auditability goals and that your team KPIs reflect the truth of overall quality status
• Serve in a capacity as a test coach to developers; reviewing unit tests and suggesting methods for better coverage

The qualifications you need:

• Strong understanding of Quality Assurance, Agile Processes and Test Automation
• Proficiency in traditional ML algorithms as well as large language models (LLMs)
• Proficiency in Model Evaluation and Validation as well as knowledge bias detection and mitigation techniques
• Knowledge of prompt engineering techniques
• Familiarity with AI/ML performance metrics
• Competence in programming languages and test automation tools (Python, PyTest, Java, RestAssured or similar), knowledge about LLM response evaluation tools (e.g. Promptfoo) will be considered a plus
• Competence with CI/CD as well as monitoring tools (e.g. DataDog)
• Strong analytical and problem-solving skills

Visa sponsorship is not offered for this role.

The base salary range below is for the role in the specified location, based on a Full Time Schedule.

Total compensation package will include base salary + bonus/commission + equity + benefits (health, dental, life, 401k, and paid time off). Please note that the base salary range is a guideline, and that the actual total compensation offer will be determined based on various factors, including, but not limited to, applicant's qualifications, skills, experiences, and location.The base salary range below is for the role in New York, based on a Full Time Schedule. $192,000—$234,000 USD

What Celonis Can Offer You:

• The unique opportunity to work with industry-leading process mining technology
• Investment in your personal growth and skill development (clear career paths, internal mobility opportunities, L&D platform, mentorships, and more)
• Great compensation and benefits packages (equity (restricted stock units), life insurance, time off, generous leave for new parents from day one, and more). For intern and working student benefits, click here.
• Physical and mental well-being support (subsidized gym membership, access to counseling, virtual events on well-being topics, and more)
• A global and growing team of Celonauts from diverse backgrounds to learn from and work with
• An open-minded culture with innovative, autonomous teams
• Business Resource Groups to help you feel connected, valued and seen (Black@Celonis, Women@Celonis, Parents@Celonis,