Senior Data Engineer (Remote)

About vidIQ

vidIQ builds software to help YouTube creators achieve their goals. Our mission is to advance the creator's journey with actionable data-driven insights and AI-powered tools. We are dedicated to our values of being creator-obsessed, lean and fast, and scientific. Having already supported millions of creators, we invite stunning new co-workers to join us in reaching millions more.

Imagine a product that reached over a million users without a traditional sales team. This product stands at the forefront of career evolution, where everyone can be their own brand with unlimited growth potential. That's the opportunity at vidIQ—an infinite market, a large and highly engaged customer base, and the chance to help scale the platform that delivers insights to millions of creators.

Discover more about vidIQ firsthand at www.vidiq.com.

About the Team

We're a team of 100+ people and growing quickly. Our vision is to be the smart copilot for every creator. We strive to create a welcoming environment where our team of smart, passionate people can share ideas with each other and do their best work. We are looking for team members who are excellent at their craft, communicate with kindness, and act like owners.

About the role

vidIQ is seeking a highly-motivated Senior Data Engineer with at least 5 years of hands-on data engineering experience to join our growing team. The ideal candidate will be a go-getter with the ability to work independently, be proactive and take full ownership. In this role, you will have oversight of working with data lake, events streaming, building ETL pipelines, wiring SQL, k8s.

You must be highly collaborative and a self-starter who is able to work in a fast-paced environment. Strong communication skills are essential in this role, as it will be integral in communicating to the back-end team where and how to implement data integration and persistence. Interest in AI and the potential it brings is a must.

An overview of our technology stack

• AWS: S3, Glue, Athena, EKS, DynamoDB, OpenSearch, MSK

• SQL, Python, Spark (on k8s), pandas, dbt, Iceberg, Airflow, Superset, Airbyte, terraform, docker, Github Actions

What you will do:

• Building efficient, critical data pipelines and data platform

• Collaborate closely with ML/AI Engineers, Backend Engineers and product to create the data sets that power vidIQ’s algorithms

• Be an advocate for data quality, acquisition of new data sources, and data infrastructure tooling

• Work closely with cross-functional teammates, including product managers, designers, backend, DevOps and ML/AI Engineers to deliver the highest impact to our users

Requirements

Job Requirements:

• 5+ years of Python experience

• Experience working with cloud; AWS preferred

• Additional experience with DynamoDB, Athena, Iceberg, S3, Glue, OpenSearch

• Familiarity with Spark preferred

• Hands-on experience with data workflow orchestration (Airflow)

• Proven track record of working with cross-functional teams in an agile-like environment

• Ability to communicate data concepts, requirements, and risks clearly to cross-functional team members, especially product analysts, data scientists, and product managers

• Preferably, you have experience developing ML-based products and services

Nice to have:

• Vector databases, embeddings, semantic search

• Real time data ingestion and analytics

• Experience or familiarity with AI technologies such as GPT, OpenAI, Langchain, Hugging Face, etc.

• K8s 

• dbt

• Backend experience

Perks and Benefits

• Work Remotely: Embrace the freedom to choose your ideal workspace anywhere in the world.

• Flexible Time Off: We support a flexible vacation policy allowing you to take time off when you need to recharge.