Senior Data Engineer - Customer Data and Insights

More than a career - a chance to make a difference in people's lives.

Build an exciting, rewarding career with us – help us make a difference for millions of people every day. Consider joining the Duke Energy team, where you'll find a friendly work environment, opportunities for growth and development, recognition for your work, and competitive pay and benefits.

Position Summary:

This position is responsible for development and consulting activities related to analytics within various Duke Energy IT and business units.

• Support or collaborate with application developers, database architects, data analysts and data scientists to ensure optimal data delivery architecture throughout ongoing projects/operations.

• Design, build, and manage analytics infrastructure that can be utilized by data analysts, data scientists, and non-technical data consumers, which enables functions of the big data platform for Analytics.

• Develop, construct, test, and maintain architectures, such as databases and large-scale processing systems that help analyze and process data in the way the Analytics organization requires.

• Develop highly scalable data management interfaces, as well as software components by employing programming languages and tools.

• Work closely with data subject matter experts to determine what data management systems are appropriate and data scientists to determine which data is needed.

• Work with stakeholders to understand the information needs and translate these into technical solutions.

• Work closely with a team of Data Science staff to take existing or new models and convert them into scalable analytical solutions.

• Design, document, build, test and deploy data pipelines that assemble large complex datasets from various sources and integrate them into a unified view.

• Identify, design, and implement operational improvements: automating manual processes, data quality checks, error handling and recovery, re-designing infrastructure as needed.

• Create data models that will allow analytics and business teams to derive insights about customer behaviors

• Build new data pipelines, identify existing data gaps and provide automated solutions to deliver analytical capabilities and enriched data to applications. 

• Responsible for obtaining data from the System of Record and establishing batch or real-time data feed to provide analysis in an automated fashion.

• Develop techniques supporting trending and analytic decision making processes 

• Apply technologies for responsive front-end experience 

• Ensure systems meet business requirements and industry practices

• Research opportunities for data acquisition and new uses for existing data

• Develop data set processes for data modeling, mining and production

• Integrate data management technologies and software engineering tools into existing structures

• Employ a variety of languages and tools (e.g. scripting languages) for integration

• Recommend ways to improve data reliability, efficiency and quality

Responsibilities:

• Develop components required for optimal extraction, transformation, and loading of data from a wide variety of data sources using a set of tools or languages (e.g. SQL) 

• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

• Implement big data analytics solutions, as well as integrate new data management technologies and software engineering tools into existing structures

• Work with stakeholders to assist with data-related technical issues and support their data infrastructure needs.

• Solve problems for Stakeholder,  construct problem frames in order to understand the analysis context and scope that will provide timely, useful results

• Create data tools for analytics and data scientist team members that assist them in building and optimizing products.

• Work in Project Teams and participate in multidisciplinary analytics project teams, translating complex functional and technical requirements into detailed architecture, design, and high performing software/techincal components

• Work with data and analytics experts to strive for greater functionality in data systems.

• Partner with business clients, establish professional relationship, and communicate with analytics clients in order to understand business needs.

Employees in this role will also:

• Translate complex functional and technical requirements into detailed archit