Staff Software Engineer, Data and AI Platforms, Central

The Chan Zuckerberg Initiative was founded by Priscilla Chan and Mark Zuckerberg in 2015 to help solve some of society’s toughest challenges — from eradicating disease and improving education to addressing the needs of our local communities. Our mission is to build a more inclusive, just, and healthy future for everyone.

The Team

Across our work in Science, Education, and within our communities, we pair technology with grantmaking, impact investing, and collaboration to help accelerate the pace of progress toward our mission. Our Central team provides the support needed to push this work forward. 

The Central team at CZI consists of our Finance, People, Real Estate, Events, Workplace, Facilities, Security, Brand & Communications, Business Systems, Central Operations, Strategic Initiatives, and Ventures teams. These teams provide strategic support and operational excellence across the board at CZI.

The Opportunity

By pairing engineers with leaders in our science and education teams, we can bring AI/ML technology and Data to the table in new ways to help drive AI powered solutions that accelerate Biomedical research and improvements in the Education space. We are uniquely positioned to design, build, and scale software systems to help educators, scientists, and policy experts better address the myriad challenges they face. We are supporting researchers and scientists around the world by developing the capacity to apply state-of-the-art methods in artificial intelligence and machine learning to solve important problems in the biomedical sciences.

The Data Engineering and Infrastructure team works on building shared tools and platforms to be used across the Chan Zuckerberg Initiative, partnering and supporting the work of an extensive group of Research Scientists, Data Scientists, AI Research Scientists, Computational Biologists, as well as a broad range of Engineers focusing on Education and Science domain problems. Members of the shared infrastructure engineering team have an impact on all of CZI's initiatives by enabling the technology solutions used by other engineering teams at CZI to scale. 

What You'll Do

• Develop and maintain the tooling and infrastructure that drives the entire data lifecycle at CZI, from ingestion and processing to secure storage and access. Your work will directly support cutting-edge research, AI model training, and data analysis across the organization.
• Partner with researchers and engineers across various domains, including genetics, imaging, and scientific literature. You'll work on various use cases, from web analytics to complex model training, ensuring data accessibility and performance for teams across CZI.
• Design and implement flexible, scalable, and performant systems to address our stakeholders’ needs, leveraging technologies like Argo Workflows and Spark for mass-scale job processing and orchestration; Vast Data, Delta Lake, and Apache Iceberg for data management and access; and cloud and on-prem HPC resources

What You'll Bring

• Education & Experience: BS, MS, or PhD in Computer Science or a related technical discipline, or equivalent experience. 7+ years of hands-on coding experience in scripting (Python, PHP, Ruby) and systems languages (Rust, C++, C#, Go, Java, or Scala).
• Data Platform Expertise: Proficiency in managing large-scale data operations, including designing scalable pipelines (streaming and batch), working with varied data types, and optimizing flexible storage solutions using tools like Argo Workflows, Spark, Delta Lake, and Apache Iceberg
• Data Management and Development Operations: Experience with data governance, metadata, and data lineage tooling like Open Lineage or Marquez. Deep experience working with building CI/CD pipelines for data infrastructure and associated observability and monitoring tooling such as Prometheus, Grafana, OpenTelemetry, Prometheus, or Honeycomb
• Supporting Complex Data and AI ML Workflows: Experience with addressing end to end data needs for working with complex data and delivering this data ready form model training, working directly with AI Researchers and AI Engineers as part of AI model training project teams. Knowledge of data pipelines that feed Deep Learning model training that makes use of accelerated computing systems (GPUs as well as ASICs, FPGAs, TPUs)
• Containerization: Extensive experience with scaling containerized applications on Kubernetes or Mesos, with a focus on secure custom containers, replicability and portability.
• Cloud & Infrastructure: Strong experience with AWS, GCP, or Azure. Exposure to hybrid environments with on-prem and colocation systems is a plus. Familiarity with Infrastructure as Code (e.g., Terraform, Ansible) and monitoring tools (Datadog, Prometheus).
• Collaboratio