Operations Manager, HITL Operations Team

About the team

AI and machine learning are at the heart of Zillow's services, from the Zestimate to personalized home recommendations to automated home tours! We believe in making all information transparent and providing insights that help more people get home - with speed, certainty and ease.

Zillow’s AI team is harnessing the power of Generative AI to build services and product experiences for a seamless and convenient real estate transaction experience. The AI Platform team is building platform services that enable usage of LLMs and multimodal models across Zillow Group businesses. We are the HITL (Human In The Loop) Operations team and one of our customers is the Rich Media Experiences team (RMX). We’re inventing the technology that is changing how people shop for their homes. Our technology gives buyers the feeling of being there from the convenience of their phones or PCs. Using Computer Vision techniques and AI-powered algorithms, we work across product, research, engineering, and business to enable people to virtually walk through a home. We believe this is the future of home shopping. And we do this as part of Zillow, within a culture that is the best most of us have ever experienced.

About the role

The HITL Operations Manager plays a critical role, leading the resources and teams that provide annotated data and insights that power our machine learning technology. This role partners closely with product management, engineering, and other stakeholders and provides human operational rigor required to scale our platform.

As an Operations Manager, you are responsible for managing data labeling, annotation programs and resources; working cross-functionally to improve existing data labeling and annotation projects; proactively surfacing insights and providing recommendations for product improvement; and actively engaging with colleagues and partners to enhance operational processes across the organization. You will be required to showcase strategic-thinking, excellent leadership, project management, problem solving, and communication to both internal and external colleagues at scale. Last, but not least, you will be responsible for identifying opportunities and implementing strategies that will help scale our business to meet future demand growth, while balancing cost constraints.

This role has been categorized as a Remote position. “Remote” employees do not have a permanent corporate office workplace and, instead, work from a physical location of their choice, which must be identified to the Company. U.S. employees may live in any of the 50 United States, with limited exceptions.

In California, Colorado, Connecticut, Hawaii, Nevada, New York, Washington state, and Washington DC the standard base pay range for this role is $75,500.00 - $120,500.00 Annually. This base pay range is specific to California, Colorado, Connecticut, Hawaii, Nevada, New York, Washington state, and Washington DC and may not be applicable to other locations.

In addition to a competitive base salary this position is also eligible for equity awards based on factors such as experience, performance and location. Actual amounts will vary depending on experience, performance and location.

Who you are

You Will Get To:

• Directly lead the day-to-day workflows and metrics of the teams responsible for Photo Localization and 3D Home Tours annotation and QA

• Identify opportunities to optimize current workflows, tooling and processes, while also identifying and implementing scalable solutions to meet near and long term production demand

• Establish team goals and work with multiple stakeholders on strategies for executing, measuring progress, and reporting results

• Own and report up on key metrics to various collaborators

• Partner with forecasting, applied science, machine learning, product management, and engineering teams to help define technical requirements for scaling solutions

• Lead contractors/outsourced relationship to ensure targets, performance metrics and SLAs are consistently met

• Work alongside our BPO to ensure training materials and coaching efforts lead to proficient Annotators who achieve optimal results

• Engage in strategic discussions to identify problems while measuring and improving the quality of workflows

Qualifications:

• 5+ years of industry experience in a product environment or operations role

• 2+ years of relevant experience in an AI/ML Operations role

• 2+ years of experience working with data visualization tools such as Tableau, or advanced Excel or Google Sheets worksheets.

• Proven experience in scaling operations in a fast-paced environment