Data Pipeline Engineer, CarOffer

Who we are

At CarGurus (NASDAQ: CARG), our mission is to give people the power to reach their destination. We started as a small team of developers determined to bring trust and transparency to car shopping. Since then, our history of innovation and go-to-market acceleration has driven industry-leading growth. In fact, we’re the largest and fastest-growing automotive marketplace, and we’ve been profitable for over 15 years.

What we do

The market is evolving, and we are too, moving the entire automotive journey online and guiding our customers through every step. That includes everything from the sale of an old car to the financing, purchase, and delivery of a new one. Today, tens of millions of consumers visit CarGurus.com each month, and ~30,000 dealerships use our products. But they're not the only ones who love CarGurus—our employees do, too. We have a people-first culture that fosters kindness, collaboration, and innovation, and empowers our Gurus with tools to fuel their career growth. Disrupting a trillion-dollar industry requires fresh and diverse perspectives. Come join us for the ride!

CarOffer fully became part of the CarGurus (NASDAQ: CARG) family in 2023, and  is an industry leader in the digital wholesale marketplace. We are automating a dealer’s ability to buy, sell, and trade cars. By bringing these two companies together, we are giving consumers and dealers alike the power to reach their destination through a full suite of digital transaction platforms. With ~14,000 dealerships in the CarOffer network, we make it easy for our dealer customers to automate inventory, connect with the right buyers, and ultimately grow their profitability.

Role overview

We are seeking a motivated and detail-oriented Data Pipeline Engineer to join our data engineering team. In this role, you will be responsible for designing, implementing, and maintaining data pipelines to efficiently ingest and stream data into our Snowflake data warehouse from diverse sources, including web analytics, third-party APIs, and FTP servers. As a key member of our team, you will play a crucial role in ensuring the reliability, scalability, and performance of our data infrastructure, enabling timely and accurate data insights for business decision-making. This position offers an exciting opportunity to work with cutting-edge technologies and industry-standard practices in data engineering while contributing to the success of our data-driven initiatives.

What you'll do

• Collaborate with stakeholders to understand data requirements and source systems, identifying data sources and defining data ingestion strategies.
• Design and implement robust data pipelines to ingest, process, and stream data into our Snowflake data warehouse from various sources, ensuring data quality and reliability.
• Develop connectors and integrations with web analytics platforms, third-party APIs, and FTP servers to automate data extraction and ingestion processes.
• Implement industry-standard practices for data pipelining, including error handling, data validation, and monitoring, to ensure data integrity and availability.
• Optimize data pipeline performance and resource utilization, identifying bottlenecks and opportunities for improvement in data processing and streaming.
• Collaborate with data engineers, software engineers, and infrastructure teams to integrate data pipelines into our data ecosystem and ensure seamless data flow.
• Document data pipeline architecture, configurations, and dependencies, ensuring clear and comprehensive documentation for future reference and troubleshooting.
• Monitor data pipeline health and performance metrics, proactively identifying and addressing issues to minimize downtime and ensure data availability.
• Stay current with emerging technologies and best practices in data engineering and streaming technologies, continuously evaluating and adopting new tools and techniques.
• Participate in team meetings, code reviews, and knowledge-sharing sessions, contributing to a collaborative and supportive team culture.

What you'll bring

• Bachelor's degree in Computer Science, Engineering, or a related field; Master's degree preferred.
• 1-3 years of experience in data engineering, software development, or a related field, with a focus on building and maintaining data pipelines.
• Hands-on experience with data pipelining tools and technologies, such as Apache Kafka, Apache NiFi, or AWS Glue, preferred.
• Proficiency in programming languages such as Python, Java, or Scala, with experience in scripting and automation.
• Familiarity with cloud platforms and services, such as AWS, Azure, or Google Cloud Platform, and experience with cloud-bas