Jobs and Careers
AM
ML Testing Process Lead, Customer Engagement Technologies
Amazon.comWashington, United StatesRemotefull_timeVerifiedPosted 19 Sept 2024
💰 $86,500/yr($40,400/yr – $86,500/yr)
About the role
We are seeking a ML (Machine Learning) Testing Process Lead to join the Omni MLDA (Machine Learning Data Associate) team within Customer Engagement Technologies (CET) to help manage the testing process of new feature/product launches for Customer Service (CS) Large Language Models (LLM) in both pre-production and post-production environments.
The CET organization powers CS by developing elegant customer and CS Associate (CSA) facing products globally. These products offer effortless self-service and automation solutions to our customers. If the customers prefer to interact with a human, we enable CSAs to effectively and elegantly solve customers’ issues using our associate facing products powered through human centered design.
Key job responsibilities
- Review intakes and understand stakeholders' requests, in collaboration with the Testing Program Manager.
- Develop SOPs and templates based on the testing plan provided by the stakeholders.
- After revising the SOPs/templates, if needed, request final sign-off on the SOP and test template.
- Present the request to the Testing Team Manager (TM) and Quality team to initiate preparation for MLDA testing support.
- Up-skill reviewers from the Quality team and testers on the new testing cases.
- Support the TM and Quality team by addressing project-related questions as they arise.
- Ensure that the Testing Team, TM, and Quality team stay on track for delivery according to SLAs.
- Communicate with stakeholders if potential blockers or delays are encountered.
- Oversee the compilation of all testing sessions.
- Draft an analysis or summary of findings, depending on stakeholder needs, once the Quality team completes their checks.
A day in the life
Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include:
1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options
3. Paid Time Off (PTO)
4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you!
At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply!
About the team
Omni MLDA is a team that remotely works to develop and fine tune generative AI machine learning models and execute testing plans to take the customer service experience to the next level. This is executed by teams of MLDAs along science teams to release localized customer-facing and agent-facing CS solutions while allowing them to focus on the development of their new products. In addition, teams with existing CS experiences can rely on the Omni MLDA Team to keep their solutions up-to-date and push identified technical defects to tech partner teams or defect elimination teams.
- Experience in project management, with a strong emphasis on collaborating with cross-functional teams
- Strong communication skills to facilitate effective information flow between testers, science, tech, and product/program teams
- Proven experience in creating and managing annotation processes, testing models, and applying quality assurance methodologies
- Analytical skills, with the ability to identify patterns, inconsistencies, and areas for improvement
- Ability to dive deep into investigating misalignments between annotations and SOPs, and identifying root causes of inaccuracies
- Experience with annotation
- Experience with project management and stakeholder management
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $40,400/year in our lowest geographic market up to $86,500/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addi
The CET organization powers CS by developing elegant customer and CS Associate (CSA) facing products globally. These products offer effortless self-service and automation solutions to our customers. If the customers prefer to interact with a human, we enable CSAs to effectively and elegantly solve customers’ issues using our associate facing products powered through human centered design.
Key job responsibilities
- Review intakes and understand stakeholders' requests, in collaboration with the Testing Program Manager.
- Develop SOPs and templates based on the testing plan provided by the stakeholders.
- After revising the SOPs/templates, if needed, request final sign-off on the SOP and test template.
- Present the request to the Testing Team Manager (TM) and Quality team to initiate preparation for MLDA testing support.
- Up-skill reviewers from the Quality team and testers on the new testing cases.
- Support the TM and Quality team by addressing project-related questions as they arise.
- Ensure that the Testing Team, TM, and Quality team stay on track for delivery according to SLAs.
- Communicate with stakeholders if potential blockers or delays are encountered.
- Oversee the compilation of all testing sessions.
- Draft an analysis or summary of findings, depending on stakeholder needs, once the Quality team completes their checks.
A day in the life
Amazon offers a full range of benefits that support you and eligible family members, including domestic partners and their children. Benefits can vary by location, the number of regularly scheduled hours you work, length of employment, and job status such as seasonal or temporary employment. The benefits that generally apply to regular, full-time employees include:
1. Medical, Dental, and Vision Coverage 2. Maternity and Parental Leave Options
3. Paid Time Off (PTO)
4. 401(k) Plan If you are not sure that every qualification on the list above describes you exactly, we'd still love to hear from you!
At Amazon, we value people with unique backgrounds, experiences, and skillsets. If you’re passionate about this role and want to make an impact on a global scale, please apply!
About the team
Omni MLDA is a team that remotely works to develop and fine tune generative AI machine learning models and execute testing plans to take the customer service experience to the next level. This is executed by teams of MLDAs along science teams to release localized customer-facing and agent-facing CS solutions while allowing them to focus on the development of their new products. In addition, teams with existing CS experiences can rely on the Omni MLDA Team to keep their solutions up-to-date and push identified technical defects to tech partner teams or defect elimination teams.
Basic Qualifications
- Experience in project management, with a strong emphasis on collaborating with cross-functional teams
- Strong communication skills to facilitate effective information flow between testers, science, tech, and product/program teams
- Proven experience in creating and managing annotation processes, testing models, and applying quality assurance methodologies
- Analytical skills, with the ability to identify patterns, inconsistencies, and areas for improvement
- Ability to dive deep into investigating misalignments between annotations and SOPs, and identifying root causes of inaccuracies
Preferred Qualifications
- Bachelor's Degree- Experience with annotation
- Experience with project management and stakeholder management
Amazon is committed to a diverse and inclusive workplace. Amazon is an equal opportunity employer and does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status. For individuals with disabilities who would like to request an accommodation, please visit https://www.amazon.jobs/en/disability/us.
Our compensation reflects the cost of labor across several US geographic markets. The base pay for this position ranges from $40,400/year in our lowest geographic market up to $86,500/year in our highest geographic market. Pay is based on a number of factors including market location and may vary depending on job-related knowledge, skills, and experience. Amazon is a total compensation company. Dependent on the position offered, equity, sign-on payments, and other forms of compensation may be provided as part of a total compensation package, in addi
Apply for this role
Generate a tailored application kit with a matched cover letter, interview prep, and CV highlights — in under 60 seconds.
Apply Now →Generate Application KitFree account required — sign up in 30s