Senior DataOps Engineer - Artificial Intelligence

Senior DataOps Engineer - Artificial Intelligence Location New York Business Area Engineering and CTO Ref # 10042607

Description & Requirements

Bloomberg’s Engineering AI department has 300+ AI practitioners building highly sought-after products and features that often require novel innovations. We are investing in AI to build better search, discovery, and workflow solutions using technologies such as transformers, gradient boosted decision trees, large language models, and dense vector databases. We are expanding our group and seeking highly skilled individuals who will be responsible for contributing to the team (or teams) of Machine Learning (ML) and Software Engineers that are bringing innovative solutions to AI-driven customer-facing products. 
At Bloomberg, we believe in fostering a transparent and efficient financial marketplace. Our business is built on technology that makes news, research, financial data, and analytics on over 35 million financial instruments searchable, discoverable, and actionable across the global capital markets.
Bloomberg has been building Artificial Intelligence applications that offer solutions to these problems with high accuracy and low latency since 2009. We build AI systems to help process and organize the ever-increasing volume of structured and unstructured information needed to make informed decisions. Our use of AI uncovers signals, helps us produce analytics about financial instruments in all asset classes, and delivers clarity when our clients need it most.We are looking for Senior DataOps Engineers with strong expertise in developing data-intensive systems and a passion for building Data Platforms for ML at scale.
As a Senior DataOps Engineer, you will be responsible for designing and building scalable, robust, and secure Data Platforms that streamline data management for AI and improve the ML model development lifecycle. We are building platform solutions to create, store, access, and manage ML artifacts such as models, datasets, features, and usage logs. Our teams extensively use open source technologies such as OCI registry, Spark, Iceberg, Flink, Kafka, S3, Hadoop, Kubernetes, Argo, Buildpacks, and other distributed data processing and cloud-native MLOps technologies.
We will trust you to:

• Design, build, and maintain multi-tenant, managed AI Data Platforms that help users ingest billions of data points using batch and streaming data processing engines into storage systems like data lakehouses, vector DB, and low latency caches.
• Work with ML and Data Engineers to understand their data needs, provide platform support, identify inefficiencies in their data practices, and set the roadmap for AI Data Platforms.
• Collaborate with open-source communities and AI application teams to build a cohesive AI Data Platform experience.
• Develop systems that provide deep provenance and traceability for data artifacts.
• Operate with understanding of data security, compliance, and integrity.

We'd love to see:

• Prior experience with data technologies such as Spark, Iceberg, Redis, or Flink.
• Experience working with ML Feature Stores such as Feast, Feathr, Databricks, Tecton, or AWS Sagemaker Feature Store.
• Working knowledge of cloud-native technologies such as Kubernetes, Argo Workflows, or Buildpacks.
• Experience contributing to or maintaining an open-source project.
• Experience working with or developing cloud AI Platforms.

You will need to have:

• 4+ years of experience working with an object-oriented programming language (Python, Go, etc).
• A Degree in Computer Science, Engineering, Mathematics, similar field of study or equivalent work experience.
• Working understanding of ML fundamentals and the role of data within the MDLC and MMLC.
• An understanding of Computer Science fundamentals such as data structures and algorit