Senior Data Engineer – Platform Foundation

The Senior Data Engineer – Platform Foundation is a hands-on, senior-level contributor embedded in the Foundations squad. You will design, build, and evolve the shared ingestion platform that underpins data delivery across the company. The platform is the product — your job is to make it reliable, extensible, and easy for other teams to adopt. 

 

The Foundations squad operates across three pillars: simplifying the overall data platform landscape by reducing complexity and consolidating redundant patterns; enabling structured and unstructured data ingestion at scale; and supporting the exposure of data products to consumers across the organization. You contribute to all three — making architectural decisions, writing production code, and enabling other teams through documentation and hands-on support. 

 

Key Responsibilities:

 

Platform Foundation Development 

• Design and implement reusable ingestion components using dlt and dbt-core, covering both structured and unstructured data sources, handling high-volume, append-heavy, and schema-drifting patterns 

• Own the Airflow platform end-to-end: extend and maintain DAGs and shared operators, handle deployments and version upgrades, and provide hands-on support to consuming teams 

• Ensure incremental loading strategies, data quality checks, and lineage metadata are first-class outputs of every pipeline 

 

Platform Simplification & Architecture 

• Identify and eliminate redundant ingestion patterns across consuming teams, drive standardization onto shared Platform Foundation components 

• Collaborate with Solution Architects to evolve the platform architecture in response to new data sources and shifting business requirements 

• Support data product exposure: define and implement governed interfaces that make data reliably accessible to internal consumers 

• Contribute to Terraform-managed infrastructure; participate in multi-cloud (AWS / Azure) deployment patterns 

 

AI Tooling & Developer Productivity 

• Actively use and evaluate AI-assisted development tools (GitHub Copilot, Claude Code, etc.) to accelerate platform Foundation delivery 

• Champion AI tooling adoption within the squad; share best practices and guardrails around AI-generated code review 

• Explore AI-powered capabilities (RAG pipelines, LLM-assisted data cataloguing) for internal platform documentation and self-service enablement 

 

DevOps & Reliability 

• Maintain and improve CI/CD pipelines (TeamCity, GitHub Actions) for platform Foundation components 

• Define and enforce observability standards: DAG/Task-level alerting, SLA tracking 

• Participate in on-call rotation for critical ingestion pipelines; drive post-incident improvements 

 

Team Enablement & Stakeholder Management 

• Produce platform Foundation documentation, runbooks, and enablement materials for consuming squads 

• Translate ambiguous or moving business requirements into concrete technical designs — comfortable challenging scope when needed 

• Mentor mid-level engineers; participate in hiring and technical assessments 

Qualifications

Basic Qualifications: 

• Bachelor's degree in Business, Inf