See yourself at Twilio

Join the team as our next Senior Software Engineer, Storage 

Who we are & why we’re hiring

Twilio powers real-time business communications and data solutions that help companies and developers worldwide build better applications and customer experiences.

Although we're headquartered in San Francisco, we have presence throughout South America, Europe, Asia and Australia. We're on a journey to becoming a globally anti-racist, anti-oppressive, anti-bias company that actively opposes racism and all forms of oppression and bias. At Twilio, we support diversity, equity & inclusion wherever we do business. We employ thousands of Twilions worldwide, and we're looking for more builders, creators, and visionaries to help fuel our growth momentum.

About the job

This position is needed to create high-quality, modern solutions to complex engineering problems and help define the future of Modern Data Warehouse and Customer Data Platforms. Our mission is to help businesses unlock customer insights and make better decisions faster and cheaper by creating a suite of industry-leading storage and compute services inside the Segment customer data platform. 

In this role, you will be responsible for designing and building a suite of storage and compute building blocks that form the foundation of a strong Customer Data Platform capable of processing billions of events per hour and beyond. You’ll also be responsible for mentoring, sharing knowledge, and guiding the technical decisions of the team to set us up for long-term success, both as a product and as a team.

Responsibilities

In this role, you’ll:

• Design and build the next generation of data lake that enable internal teams to unlock value from their user profiles data using Spark or Presto/TrinoDb and using the table format Iceberg
• Build globally available and highly scalable distributed systems
• Collaborate with teams across the R&D org to ensure their services are multi-region capable and be a liaison to help data scientists for building/diagnosing big data queries
• Mentor other engineers on the team on technical architecture and design
• Participate in an on-call rotation to support our business-critical infrastructure

Qualifications 

Not all applicants will have skills that match a job description exactly. Twilio values diverse experiences in other industries, and we encourage everyone who meets the required qualifications to apply. While having “desired” qualifications make for a strong candidate, we encourage applicants with alternative experiences to also apply. If your career is just starting or hasn't followed a traditional path, don't let that stop you from considering Twilio. We are always looking for people who will bring something new to the table!

 

Required:

• 5+ years experience writing production-grade code in a modern programming language.
• Solid hands on experience working with data pipelines using Apache Spark framework or technologies such as Flink, Kafka, Druid or Presto
• Well versed of file formats like Parquet, Avro, Hudi, Iceberg etc
• Ability to read and write SQL queries to pull required complex data.
• Strong theoretical fundamentals and hands-on experience designing and implementing highly available and performant fault-tolerant distributed systems.
• Fundamental understanding and experience with asynchronous architectures, their benefits and challenges including various broker implementations (Kafka, SQS, MQ)
• Experience operating large-scale, distributed systems on top of cloud infrastructure such as Amazon Web Services (AWS) or Google Compute Platform (GCP)
• Have hands-on experience with container orchestration frameworks (e.g. Kubernetes, EKS, ECS)
• Experience shipping services (products) following CI/CD development paradigm.
• Deep understanding of architectural patterns of high-scale web application (e.g., well-designed APIs, high volume data pipelines, efficient algorithms).
• Have led teams, large projects, or owned and built an important, complex system end to end.
• Excellent written and verbal communication skills.

Desired:

• Have hands-on experience with container orchestration frameworks (e.g. Kubernetes, EKS, ECS)
• Champion of good engineering habits: development techniques, security, technical debt balance

Location 

This role will be Remote - US.

What We Offer

There are many benefits to working at Twilio, including, in addition to comp