Data Engineer II - Getting Customers Ready for AI
MicrosoftAbout the role
Security represents the most critical priorities for our customers in a world awash in digital threats, regulatory scrutiny, and estate complexity. Microsoft Security aspires to make the world a safer place for all. We want to reshape security and empower every user, customer, and developer with a security cloud that protects them with end to end, simplified solutions. The Microsoft Security organization accelerates Microsoft’s mission and bold ambitions to ensure that our company and industry is securing digital technology platforms, devices, and clouds in our customers’ heterogeneous environments, as well as ensuring the security of our own internal estate. Our culture is centered on embracing a growth mindset, a theme of inspiring excellence, and encouraging teams and leaders to bring their best each day. In doing so, we create life-changing innovations that impact billions of lives around the world.
Microsoft Security's Getting Customers Ready for AI team is seeking a Data Engineer II to help build the data foundations that power AI-native security systems.
In this role, you will design and develop scalable data platforms, pipelines, and telemetry systems that transform raw signals across identity, devices, data, applications, and infrastructure into high-quality, trusted datasets for analytics and AI-driven insights.
You will work in a collaborative engineering environment to deliver reliable, secure, and performant data systems, enabling downstream AI models, security detections, and customer-facing experiences. This role is ideal for engineers looking to deepen their expertise in large-scale data systems, distributed processing, and modern data platform architectures for AI and Agents.
Microsoft’s mission is to empower every person and every organization on the planet to achieve more. As employees we come together with a growth mindset, innovate to empower others, and collaborate to realize our shared goals. Each day we build on our values of respect, integrity, and accountability to create a culture of inclusion where everyone can thrive at work and beyond.
Responsibilities
Data Platform & Pipeline Engineering
Design and build scalable data pipelines (batch and streaming) to process large volumes of security and operational data.
Develop and optimize ETL/ELT workflows that transform raw telemetry into structured, consumable datasets.
Implement data ingestion frameworks to integrate multi-source data from services, APIs, and event streams.
Improve pipeline performance, reliability, and efficiency through partitioning, indexing, and optimization techniques.
Data Modeling & Storage Systems
Design and evolve data models, schemas, and storage strategies for analytics and AI use cases.
Work with distributed storage systems (e.g., data lakes, warehouses) to ensure scalability and cost efficiency.
Maintain data partitioning, retention, and lifecycle strategies aligned to business and compliance needs.
Ensure data is structured for downstream ML pipelines, feature engineering, and analytics workloads.
Data Quality, Governance & Security
Implement data validation, quality checks, and observability frameworks to ensure data accuracy and reliability.
Apply best practices for data governance, lineage, and auditing across pipelines and datasets.
Ensure compliance with security, privacy, and regulatory requirements when handling sensitive data.
Contribute to standardization of data contracts and schemas across services.
AI Readiness & Intelligence Enablement
Enable high-quality datasets for AI/ML teams, including support for feature pipelines and training data preparation.
Collaborate with AI engineers to ensure data is optimized for RAG pipelines, model training, and evaluation workflows.
Build and maintain telemetry pipelines and metrics systems that provide insights into AI system performance and usage.
Support development of data-driven signals and insights that improve customer security posture.
Collaboration & Execution
Partner with engineering, data
Apply for this role
Generate a tailored application kit with a matched cover letter, interview prep, and CV highlights — in under 60 seconds.
Apply Now →Generate Application KitFree account required — sign up in 30s