Senior Data Engineer

If you’re passionate about building a better future for individuals, communities, and our country—and you’re committed to working hard to play your part in building that future—consider WGU as the next step in your career.

Driven by a mission to expand access to higher education through online, competency-based degree programs, WGU is also committed to being a great place to work for a diverse workforce of student-focused professionals. The university has pioneered a new way to learn in the 21st century, one that has received praise from academic, industry, government, and media leaders. Whatever your role, working for WGU gives you a part to play in helping students graduate, creating a better tomorrow for themselves and their families.

The salary range for this position takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs.

At WGU, it is not typical for an individual to be hired at or near the top of the range for their position, and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is:
 

Grade: Technical 409

Pay Range: $127,700.00 - $191,500.00

Job Description

The Sr. Data Engineer should be agnostic to tools and should be able to design, architect and code using Apache Spark and other cloud technologies.  The position will design how data will flow through hybrid data environments comprised of open-source big data platforms and traditional database systems. The core responsibilities for this position include design of data and systems architecture for Data Lake and data warehouse and projects encompassing dimensional and normalized data modeling. The Sr. Data Engineer will improve technical standards in the environment ensuring optimal use of data warehouse, Data Lake and other data stores to solve business problems. They will serve as the go-to person for query performance and data quality issues including data profiling. Sr. Data Engineer should be able to support data analysts and data scientists with requirements and be able to deliver data sources for their respective requirements

Primary Responsibilities

• Works on cloud technologies to design scalable, performant, and efficient data lake systems.

• Establishes design and methodology for the database build processes.

• Architects and designs complete data model solutions including Data Pipelines.

• Implements necessary data protection and security processes.

• Creates extract processes for Business Intelligence tool access layer.

• Translates business problems/information requirements accurately to logical/physical data models aligning the customer’s data architecture standards.

• Performs research and analysis to come up with solutions for complex business problems.

• Monitors query performance and fine tune Spark SQL queries, if required on a regular basis.

• Regularly profiles data, publishes data profiles, and takes corrective actions.

• Ensures high quality data delivery by designing process and business audits.

• Documents and describes data migration techniques used.

• Documents, reverse engineers, and analyzes data mapping using data integration code & tools.

• Works with API for data pulls and integrations of it with other systems data in the Data Lake House.

• Performs impact analysis using Data Integration/Data Virtualization tool repositories, DB data dictionary, UNIX scripts and frontend code on versioning systems.

• Analyzes and researches data on multiple platforms as well as multiple heterogeneous databases including custom developed databases.

• Positively impacts projects by completing tasks assigned on time.

• Ingests and transforms structured, semi-structured, and unstructured data from sources including relational databases, NoSQL, external APIs, JSON, XML, delimited files, and more.

• Supports data analysts and data scientists with requirements and delivers data sources for their respective requirements.

• Designs and develops validation & testing tools to analyze and debug issues.

• Performs other job-related duties as assigned.

Knowledge, Skills, and Abilities
•    Excellent verbal and written communication skills, including the ability to write technical documentation
•    Ability to work and deliver in a team environment
•    Ability to mentor less experienced engineers in data pipeline architecture and