Senior Data Engineer

HeartFlow, Inc. is a medical technology company transforming the diagnosis and management of coronary artery disease, the #1 cause of death worldwide, using cutting-edge technology. The flagship product—an AI-based, non-invasive cardiac test called the HeartFlow Analysis—provides a color-coded, 3D model of a patient’s coronary arteries indicating the impact blockages have on blood flow to the heart. It offers physicians a completely novel way to diagnose and treat cardiac patients. Our pipeline of products is growing and so is our team; join us in helping to revolutionize precision heartcare.   HeartFlow is a VC-backed, pre-IPO company that has received international recognition for exceptional strides in healthcare innovation, is supported by medical societies around the world, cleared for use in the US, UK, Europe, Japan and Canada, and has been used for more than 200,000 patients worldwide. 

The Sr. Data Engineer brings DataOps, data engineering, DevOps and cloud computing expertise in expanding and optimizing our data lake processing pipeline and data analytics architecture. They bring technical expertise to design, architect, implement and support solutions to scale and optimize data flow into and out of cloud-based data lakes for cross functional teams to consume for Tableau reporting, Ad-hoc querying and Machine learning processing. #LI-IB1; #LI-Remote

In addition, expertise in cross-team (NetOps, DevOps, InfoSec, SecOPs, etc.) collaboration via use of professional skills are utilized to perform stretch roles as Sr. Data Engineer.

The Sr. Data Engineer will be a mentor and SME to other members of the team, as well as consult to management and leadership.

Job Responsibilities: 

• Design, build, and maintain a scalable and low-latency infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources and targets using AWS data lake related services and analytics-related technologies. 
• Support and collaborate with down-stream consumers, i.e. Tableau, Ad-hoc and Machine Learning to ensure their success by assembling and delivering required data sets, security, and acceptable query times
• Perform exporting/sync of data to third-party products, i.e. Salesforce, FTP Servers, etc. using AWS AppFlow, SFTP SSH, etc.
• Be SME from within the team to provide expertise to bring innovative ideas from concept to fruition via collaborative POCs, Spikes and /or discussions with team members.
• Identify, design, document and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. 
• Understand and guide a comprehensive testing, continuous integration and development framework for schema, data, and functional processes/pipelines, etc. 
• Be SME for tools used by Data Engineering, i.e. GitHub, Jira, Confluence, etc.
• Manage AWS accounts dedicated to DataOps team with root level access that have DevOps, NetOps, InfoSec and DataOps stretch-role responsibilities. 

Skills Needed:  

• Strong experience with AWS cloud services: S3, DMS, RDS, Lambda, Redshift, Glue, AppFlow, EC2, in cross-region and cross-account implementations
• Strong experience with AWS security: IAM Roles/Policies/Users, Okta/SSO
• Strong experience with AWS networking: VPC, Subnets, security groups, route tables, gateways, etc.
• Strong experience with technical writing to document processes, best practices, create architecture diagrams, etc. using Atlassian Confluence, Lucid Charts, etc.
• Strong experience with Agile Scrum: Managing, creating and maintaining Jira board, issues, sprint shutdown, grooming, status updates with management, supporting documentation and sprint demos. Perform stretch-role of Scrum Master.
• Strong experience with Infrastructure as Code via Terraform, AWS CDK for Python, AWS CloudFormation, etc.
• Strong experience with NoSQL databases, i.e. DynamoDB, DocumentDB and/or MongoDB
• Strong experience with relational/DW SQL databases such as MySQL, PostgreSQL, and Redshift
• Strong experience with programming in Python via python.org and Anaconda python using AWS Boto3, PyCharm IDE, etc. and data engineering related packages: pandas, numpy, scipy, sqlalchemy, pyarrow, redshift connector
• Strong experience building and optimizing data pipelines, architectures, data lakes, data sets, etc. 
• Strong experience with reading and writing of multiple file formats, i.e. JSON, text/CSV, and parquet
• Strong experience with transferring and receiving of files via SFTP SSH using ssh CLI, Python, Filezilla, etc.
• Strong experience with Code repo management - Git and GitHub, managing and overseeing repos, branches, pull requests, etc. using standalone CLI