Staff Software Engineer, Distributed Storage

Airbnb was born in 2007 when two Hosts welcomed three guests to their San Francisco home, and has since grown to over 4 million Hosts who have welcomed more than 1 billion guest arrivals in almost every country across the globe. Every day, Hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way.

The Community You Will Join:

The Storage Foundation team within Airbnb’s Storage Infra org owns Airbnb's online serving storage system. The group is responsible for designing, building and operating the software and solutions listed below. With users around the world, reliability, scalability, efficiency, high availability and platform evolution are our team’s core concerns.

1. A new source-of-truth, open-source NewSql database built on top of stateful Kubernetes that hosts all the critical Airbnb user, listing and financial data, with all the essential DB capabilities on such as backup restore, CDC, multi-tenancy. This will also serve as the unified storage backend for Airbnb Infrastructure.
2. A set of managed storage services / APIs built on top of the unified backend, e.g.:
• A highly available, low-latency, distributed, multi-tenant K/V storage layer for Airbnb user profiling, Search, ML, pricing teams, supporting million-level read QPS, 99.9+% availability across thousands of customer tables.
• Design and deliver a new paved path for online Graph Store that would serve as a source-of-truth Graph database, as well as a low-latency, highly available querying engine powering Airbnb’s Knowledge and Social Graph.
3. A set of control planes and provisioning managements for OpenSearch and ElasticCache for hundreds of Airbnb production clusters, at million+ IOPS and million+ indexing QPS.

The Difference You Will Make:

We’re looking to add Staff Engineers who are hands-on and capable of solving broad technical challenges the team has to tackle. As a seasoned technical contributor / leader, you will bring your technical expertise, experience and thought leadership to our organization, and start in either working groups below:

1. Being part of Core DB team that builds out all the essential DB capabilities such as stateful deployment of database on Kubernetes, backup restore and disaster recovery, CDC, snapshotting, quality-of-service on a multi-tenant cluster; and then migrate existing Airbnb critical DBs to the new stack. In the longer term advance Airbnb infra’s capability towards multi-Cloud and multi-Region.
2. Being part of the Managed Storage Services team (1) to complete building V2 of our KV store on top of open-source NewSql database, and adding features like multi-tenant cost attribution, large-scale bulk loading, etc, (2) to participate in driving the roadmap, implement and design next set of fully managed storage services being a distributed caching, graph storage and/or pubsub, etc. 

A Typical Day: 

• Build a fault-tolerant ingestion layer on top of open-source NewSql database to ingest TBs of data with minimal impact on online traffic.
• Work with AWS to provide a secure client that complements Airbnb’s mesh layer and manage hundreds of ElasticCache customers cost-effectively.
• Design and implement a seamless database migration pipeline where consistency, latency and correctness are of the utmost importance.
• Work within Storage Infra and with Knowledge Graph and Search team to design and implement bulk loading and transactional API for writes, as well as efficient serving strategies around materialized views. 
• Participate in regular on-call and interact with key customers to understand their requirements and pain points.
• Coach more junior members of the team and guide them on iterating on designs and solving the right problems, provide insightful design and code review feedback

Your Expertise:

• 9+ years of relevant industry experience in a fast paced, high growth tech environment.
• Knack for writing clean, readable, testable, maintainable code; experience in deep diving and then owning a complex code base.
• Hands-on experience in building and operating distributed storage and services.
• Ability to understand and decompose large-scale distributed systems and figure out monitoring metrics, failure scenarios and debug them in an efficient manner.
• Strong collaboration and communication skills with a customer-first mindset, ability to abstract various customer requests into a platform vision and solution.
• Experience in designing architecture that is long-term and evolvable.
• Knowledge of Kubernetes, public cloud platforms (AWS, ElasticSearch, ElasticCache, Google Cloud Platform, etc) and open source preferred but not strictly required.