Sr Staff Data Engineer

Company Description

NBCUniversal is one of the world's leading media and entertainment companies. We create world-class content, which we distribute across our portfolio of film, television, and streaming, and bring to life through our theme parks and consumer experiences. We own and operate leading entertainment and news brands, including NBC, NBC News, MSNBC, CNBC, NBC Sports, Telemundo, NBC Local Stations, Bravo, USA Network, and Peacock, our premium ad-supported streaming service. We produce and distribute premier filmed entertainment and programming through Universal Filmed Entertainment Group and Universal Studio Group, and have world-renowned theme parks and attractions through Universal Destinations & Experiences. NBCUniversal is a subsidiary of Comcast Corporation.

Our impact is rooted in improving the communities where our employees, customers, and audiences live and work. We have a rich tradition of giving back and ensuring our employees have the opportunity to serve their communities. We champion an inclusive culture and strive to attract and develop a talented workforce to create and deliver a wide range of content reflecting our world.

Comcast NBCUniversal has announced its intent to create a new publicly traded company ('Versant') comprised of most of NBCUniversal's cable television networks, including USA Network, CNBC, MSNBC, Oxygen, E!, SYFY and Golf Channel along with complementary digital assets Fandango, Rotten Tomatoes, GolfNow, GolfPass, and SportsEngine. The well-capitalized company will have significant scale as a pure-play set of assets anchored by leading news, sports and entertainment content. The spin-off is expected to be completed during 2025.

Job Description

We are seeking a Senior Staff Data Engineer looking to build the next generation of data pipelines and applications across the development of innovative new systems and solutions using a rapidly changing landscape of emerging technologies including generative AI and large language models. Working across the practices, techniques and tools used for the operational management of large language models in production environments – the Senior Staff Data Engineer role is proper for you if you're a subject matter expert in designing data integration frameworks and pipelines and still love to jump in and be "hands-on" when needed. This team is focused on proving the value of new tech and bringing it to production quickly.

In the Senior Staff Data Engineer role, you'll have the opportunity to partner with internal stakeholders, data engineers, visualization experts, data scientists, and other technologists across the businesses. You've come to the right place if you love to take large, disparate data sets and build them into flexible and scalable analytics applications and warehouses. Here, you can create the extraordinary. Join us. The ideal candidate should be well-versed in designing, building, and supporting APIs, machine learning services and frameworks, LLMs, lang-chain, and foundational data warehousing technologies. In addition, the candidate should be excited about the possibilities that Generative AI can be leveraged to accelerate various parts of the business.

Your primary focus will be building reliable, scalable, and efficient pipelines for use in LLMs and crafting our vision for LLM analytics. You will be essential in defining the team's strategy, evaluating, and integrating data patterns and technologies, and building pipelines alongside domain experts and data scientists.

Responsibilities:

• Design, build, and scale data pipelines across a variety of source systems and streams (internal, third-party, and cloud-based), distributed/elastic environments, and downstream applications and self-service solutions.
• Deep understanding of Machine Learning best practices (e.g., training/serving, feature engineering, feature/model selection, imbalance data, RAG patterns) and algorithms (e.g., deep learnings, optimization)
• Solid understanding of data modeling, warehousing, and architecture principles.
• Implement appropriate design patterns while optimizing performance, cost, security, and scale and end-user experience.
• Collaborate with cross-functional teams to understand data requirements and develop efficient data acquisition and integration strategies.
• Interface with other technology teams to extract, load, and transform data from a wide variety of data sources using cloud-native data engineering principles.
• Become a subject matter expert for data engineering-related technologies and designs.
• Coach and guide others within the organization to build scalable pipelines based on foundational data engineering principles.
• Participate in development sprints, demos, and retrospectives alongside releases and deployment.
• Build and manage relationships with supportin