DHTS IT ANALYST - Sr. Data Engineer

At Duke Health, we're driven by a commitment to compassionate care that changes the lives of patients, their loved ones, and the greater community. No matter where your talents lie, join us and discover how we can advance health together.
About Duke Health Technology Solutions
Pursue your passion for caring and innovation with Duke Heath Technology Solutions, which is dedicated to the transformation, development, and management of enterprise information technology solutions across Duke Health. By harnessing the power of innovative technologies like cloud computing and artificial intelligence — and pairing them with a forward-thinking approach — Duke Health Technology Solutions is revolutionizing the future of health care at Duke Health and beyond.

POSITION DESCRIPTION

Job Title:  IT Developer II (Data Engineer)

 

DHTS: Duke Health Technology Solutions (DHTS) is a robust, specialized division of Duke University Health System dedicated to the development and management of enterprise IT systems. A 2018 ‘Most Wired’ health system, Duke is nationally recognized for IT and information management as the first healthcare system to achieve the Davies Award – highest honor by the Healthcare Information and Management Systems Society (HIMSS) – for inpatient, ambulatory, and analytics health information technology capabilities. Duke was also the first to achieve Stage 7 HIMSS for Hospital EMR, Ambulatory EMR, and Analytics adoption. Our employees are among the top-skilled IT experts in the Triangle. We partner with leading scholars, clinicians, and researchers across Duke University and Duke Health to develop innovative technologies that support our mission of delivering tomorrow's healthcare today.

 

Primary Role:  This position reports to the DHTS-Data Partnerships, Director of Data and Analytics Platforms.  This individual will be primarily responsible for the development of data integration and delivery pipelines while also expanding the FHIR-based content stored within Duke’s Enterprise Analytics Platform.  These solutions should capitalize on technologies to improve the value of analytical data, improve the effectiveness of information stewardship, and streamline the flow of data in the organization.  Solutions should focus on using state-of-the-art data and analytics tools including traditional and near real-time data warehousing, big data, relational and document-based databases using both extract, load, transform (ELT) toolsets as well as REST APIs and FHIR.  The ideal candidate will be comfortable with data science platforms with proven experience leveraging DevOps and Automation/Orchestration tools.

 

Essential Tasks/Responsibilities

• Create and maintain optimal data pipeline architecture.
• Develop a data lake on Microsoft Azure using the medallion architecture leveraging a delta lake format for the silver layer.
• Assemble large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Recommend design of analytics solutions that improves data integration, data quality, and data delivery with an eye toward re-useable components
• Articulate differences, advantages, and disadvantages between architectural solution methods
• Work with Agile team members to document and execute test plans for data loading and data validation scripts. Support the code promotion process through development and production as required by using standard CI/CD processes.
• Develop, implement, and maintain schedule/dependency logic for automated ETL processing.
• Develop monitoring, logging, and error notification processes to ensure data is updated as expected and processing metrics reported.
• Participate in the creation and maintenance of standards for coding, documentation, error handling, error notification, logging, etc.
• Accountable for conforming to established architectural, developmental, and operational standards and practices including the creation of metadata.
• Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data, and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Evaluate and recommend database development tools
• Assist in application and database performance tuning
• Participate in system architecture design
• Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry