Senior Software Engineer, Data Engineering

Airbnb was born in 2007 when two hosts welcomed three guests to their San Francisco home, and has since grown to over 5 million hosts who have welcomed over 2 billion guest arrivals in almost every country across the globe. Every day, hosts offer unique stays and experiences that make it possible for guests to connect with communities in a more authentic way.

The Community You Will Join:

We are currently hiring for multiple teams: 

Payments: We are building a world-class payments platform — one that currently supports 190+ countries and regions, 70+ currencies, connects dozens of payment providers and banks, and processes multiple billions of dollars.  As the platform grows, we’ll be adding new payment partners, global licenses, and building new payment experiences for our guests and hosts.  The Payments team builds a scalable foundation to support global scale, helps the company grow by bringing new markets and demographics to the platform, and enables new business initiatives to thrive by providing easy to use payment services.

Apps and Compliance: This team is responsible for building scalable, high quality data sets and solutions to enable Airbnb to comply with Tax, payments and legal regulations to ensure business continuity. In addition, this team is responsible for building data sets for Airbnb’s internal applications (e.g. CRM data, projects data, workspace data) to fuel growth and drive operational efficiencies. 

The Difference You Will Make:

Payments: As Airbnb expands its presence, it faces a host of unique challenges: managing this large volume of data, deriving information and insights from it, using it to make informed business decisions, and creating data-powered products. We also want to ensure that our financial data has high fidelity, is available in a timely fashion, is comprehensive, is organized in a way that is easy to understand, and is easily discoverable. This problem is more relevant today, as our entire payments infrastructure is undergoing a major overhaul, with systems being redesigned in a service-oriented architecture.

Apps and Compliance: Our team charter is on enabling Airbnb to comply with Tax, Payments, and Legal regulations so that our Hosts can continue to operate in regulated geos. Our products ingest, process, validate, and deliver large datasets to government authorities (often partnered with tax remittance) so our data must be of the highest accuracy and quality. As you build and maintain key components of the critical Compliance data ecosystem, you’ll have the opportunity to contribute to creating standards and best practices for Airbnb’s Data Engineering. Your work on solving complex business challenges at scale will be instrumental in shaping the tools, processes, and standards used by the broader data community.

A Typical Day:

• Design, build, and maintain robust and efficient data pipelines that collect, process, and store data from various sources, including user interactions, listing details, and external data feeds.

• Develop data models that enable the efficient analysis and manipulation of data for merchandising optimization. Ensure data quality, consistency, and accuracy.

• Build scalable data pipelines (SparkSQL & Scala) leveraging Airflow scheduler/executor framework

• Collaborate with cross-functional teams, including Data Scientists, Product Managers, and Software Engineers, to define data requirements, and deliver data solutions that drive merchandising and sales improvements.

• Contribute to the broader Data Engineering community at Airbnb to influence tooling and standards to improve culture and productivity

• Improve code and data quality by leveraging and contributing to internal tools to automatically detect and mitigate issues

Your Expertise:

• 5-9+ years of relevant industry experience with a BS/Masters, or 2+ years with a PhD

• Experience with distributed processing technologies and frameworks, such as Hadoop, Spark, Kafka, and distributed storage systems (e.g., HDFS, S3)

• Demonstrated ability to analyze large data sets to identify gaps and inconsistencies, provide data insights, and advance effective product solutions

• Expertise with ETL schedulers such as Apache Airflow, Luigi, Oozie, AWS Glue or similar frameworks

• Solid understanding of data warehousing concepts and hands-o