Senior Data Engineer

About Agero:

Wherever drivers go, we’re leading the way. Agero’s mission is to rethink the vehicle ownership experience through a powerful combination of passionate people and data-driven technology, strengthening our clients’ relationships with their customers. As the #1 B2B, white-label provider of digital driver assistance services, we’re pushing the industry in a new direction, taking manual processes, and redefining them as digital, transparent, and connected. This includes: an industry-leading dispatch management platform powered by Swoop; comprehensive accident management services; knowledgeable consumer affairs and connected vehicle capabilities; and a growing marketplace of services, discounts and support enabled by a robust partner ecosystem. The company has over 150 million vehicle coverage points in partnership with leading automobile manufacturers, insurance carriers and many others. Managing one of the largest national networks of service providers, Agero responds to approximately 12 million service events annually. Agero, a member company of The Cross Country Group, is headquartered in Medford, Mass., with operations throughout North America. To learn more, visit https://www.agero.com/.

POSITION SUMMARY:

We are seeking a Data Engineer who is passionate about data and eager to make a meaningful impact. In this role, you will design, build, and maintain the core data infrastructure that powers our analytics, machine learning, and data science initiatives. Your responsibilities will include optimizing data management processes, ensuring data quality and reliability, and developing scalable, efficient data models to support advanced analytics and data-driven decision-making. Success in this role requires a strong technical foundation, a collaborative mindset, and a drive to deliver innovative and impactful solutions.

ESSENTIAL FUNCTIONS:

• Data Pipelines:
• Develop and maintain robust ETL/ELT pipelines to ingest data from diverse sources (relational and NoSQL databases, APIs, etc.), including implementing best practices for real-time and batch data ingestion.
• Create and optimize data workflows using modern orchestration tools (e.g., Apache Airflow, Snowflake Tasks, Dagster, Mage).
• Cloud Cost Optimization:
• Monitor and optimize cloud costs (e.g., AWS, Snowflake) by analyzing resource usage and implementing cost-saving strategies.  
• Perform query optimization in Snowflake to reduce compute costs and improve performance.
• Data Foundations:
• Develop and maintain modern data architectures, including data lakes and data warehouses (e.g., Snowflake, Databricks, Redshift), considering trade-offs of different data storage solutions and ensuring alignment with business requirements and SLAs.
• Data Modeling & Transformation:
• Apply dimensional modeling techniques (Kimball), star and snowflake schemas, and normalization vs. denormalization strategies based on use cases.
• Develop transformations using DBT (Core or Cloud), Spark (PySpark), or other frameworks.
• Collaborate on emerging approaches such as data mesh or specialized templates (e.g., Jina) to handle complex data needs.
• Coding:
• Write reusable, efficient, and scalable code in Python, PySpark, and SQL.
• Integrate serverless computing frameworks or modern API frameworks to support data-driven applications (FastAPI, Flask).
• Develop and maintain data-intensive UIs and dashboards using tools like Streamlit, Dash, Plotly, or React.
• Data Quality Control:
• Establish data governance and data quality frameworks, using either custom solutions or popular open-source/commercial tools (e.g., DBT tests, Great Expectations, Soda).  
• Implement data observability solutions to monitor and alert on data integrity and reliability (e.g., Monte Carlo, Alation, or Elementary).
• Define SLAs, SLOs, and processes to identify, troubleshoot, and resolve data issues.
• Teamwork:
• Work cross-functionally with data scientists, analysts, and business stakeholders to translate requirements into robust data solutions.
• Follow and advocate for best practices in version control, CI/CD.
• Document data flows, processes, and architecture to facilitate knowledge sharing and maintainability.

REQUIREMENTS:

EDUCATION &am