Lead Data Engineer

Lead Data Engineer
Prog Analyst Spec
Research Administration Information Services (RAIS)

   

Lead Data Engineer

Cornell University is one of the leading research universities in the world. Established in 1865 as an “institution where any person can find instruction in any study”, we pride ourselves on the diversity of our student body, our faculty and staff, and our academic disciplines. The Research Division at Cornell plays an important role in making our diverse and groundbreaking research programs come to life. We help our world class research community to find funding, navigate complex regulations, and bring new inventions to the market. Our Information Services team makes this possible by providing research management systems that are easy for the research community to use based on excellent user-experience research and design. They also build the data infrastructure necessary to bring together all of the data needed for reporting on the success of our research efforts.

Cornell University embraces diversity and seeks candidates who will contribute to a climate that supports students, faculty and staff of all identities and backgrounds. We strongly encourage individuals from underrepresented and/or marginalized identities to apply.

  

The Information Services team is looking for talented and creative individuals to join us in our quest to make it easier for the Cornell research community to manage their research. We consider our environment to be its own research laboratory where team members are encouraged to voice new ideas and try new approaches. We’re early adopters in the use of new technologies that can help us to automate development and deployment of the application and data products we build or integrate with. There’s no doubt that you’ll be able to learn and grow as part of our team. We have opportunities for advancement in our group and within the university. The ability to take classes or pursue a degree through our employee degree program is also available and encouraged. Cornell benefits are hard to beat, including a contribution of 10% of your salary to your retirement plan pre-tax. 

The Lead Data Engineer position is a senior member of the Information Services team, responsible for all of our technical decisions, including our technology stack and data architecture. The position reports to the Director of Information services and provides day-to-day technical leadership for the data team.

What You Will Do:

• Design and maintain our data infrastructure that gets data to and from the data products we create and the applications and data sources we integrate with
• Review our tool chain to make sure we’re using the best apps and practices
• Help build and lead our data team; implement procedures and work instructions
• Work on new data projects - help the project and product management teams understand what’s possible and figure out the best way to deliver; break down tasks for the data team
• Write code, e.g. Python for data, Terraform for DevOps
• Prioritize the development backlog with the product team - tell them when you need time to refactor or prioritize “technical debt” items that pose a risk
• Infrastructure design, automation, and maintenance in cloud and on-premises hosting environments (current setup - Terraform: Infrastructure-as-code, CodeBuild: Build tools, Docker/ECS/ECR: containerization, Fargate: Serverless tasks, CloudWatch/Prometheus: Monitor/Log/Trigger, S3/RDS: Storage).
• Troubleshoot potential data issues reported with production reporting systems and integrations
• Create and test data visualizations to aid understanding of research administration data

           

What You Will Need:

• 5 or more years working in a data analyst, data engineer, or other data science role
• Bachelor’s degree or higher
• Advanced knowledge of SQL
• Experience with Python data science ecosystem (Pandas, Jupyter Notebooks, etc.)
• Experience with DevOps, specifically AWS, Terraform, and Docker
• Thorough understanding of relational database systems and design
• Thorough understanding of non-traditional/NoSQL data formats and how to work with them (json, xml, csv)
• Thorough understanding of popular data warehouse data structures (e.g., star, snowflake)
• Ability to use tools to introspect data sources to understand cardinality, relationships, data quality or potential issues
• Meticulous attention to detail and documentation habits
• Excellent communication and organization skills
• Good team player with an ability to work in a cross-functional team environment

What We Prefer:

• Bachelor’s degree or higher with a concentration in a quantitative field such as fina