Senior Data Engineer

The Senior Data Engineer at Workiva will be an instrumental part of data workflows throughout the organization. You will own the technical design and implementation of systems that support multiple data analytics teams and business intelligence engineers reliably and at scale using AWS cloud environments. You will provide cutting-edge, reliable, and easy-to-use systems for ingesting and processing data and help the teams that build data-intensive applications be successful. 

This role will collaborate with many cross-functional teams on the planning, execution, and successful completion of technical projects with the ultimate purpose of improving customer experience. You will build and maintain batch and real-time data flows used for business intelligence, analytics, and machine learning within all organizations across Workiva. This also involves storing and exposing data via a Database, Data Lake, and other APIs. Senior Data Engineers work primarily with other Data Engineers but also with Data Scientists, ML Engineers, and business partners to ensure quality, reliability, and performance at the highest level.

What You'll Do

• Own the implementation of data pipelines from various data sources using new and existing patterns

• Design systems that enable next-generation AI to unlock business insights and sales opportunities

• Maintain the health of the data ecosystem by configuring deployment, monitors, defining alerts on common failure points, and giving feedback on data quality to data owners and business partners

• Build highly reliable CI/CD processes to ensure high quality data throughout the data ecosystem

• Review peer code and submit thorough and actionable feedback based on team standards and industry best practices

• Triage and resolve production issues. Communicate with individual business partners on status and escalate as needed

• Design data lake storage and access patterns to match customer requirements and conform to naming standards

• Understand the data at a deep level, apply security appropriately, and escalate as needed

• Tune processes and SQL to reduce cost and wait time. Implement systems to balance data volume, latency and customer requirements

• Stay up-to-date with emerging technologies and industry trends in data engineering, and recommend innovative solutions to enhance our data infrastructure

• Work with business partners to write requirements and test deployed code

• Join rotation to support production workflows during off hours

What You'll Need

Minimum Qualifications

• 4+ years of relevant experience in the data engineering role, including data warehousing and business intelligence tools, techniques, and technology, or experience in analytics, business analysis or comparable consumer analytics solutions

• Undergraduate Degree or equivalent combination of education and experience in a related field

 Preferred Qualifications

• Bachelor’s degree in Computer Science, Engineering, Math, Finance, Statistics or related discipline

• Extensive experience with cloud data warehouses such as Redshift, Snowflake, and Databricks

• In-depth knowledge and understanding of Data Lake design and principles

• Experience in big data processing and using databases in a business environment with large-scale, complex datasets. (SQL, Hadoop, Spark, Flink, Beam etc) and the tools to manage and interact with data (Airflow, DBT, Fivetran etc)

• Experience using cloud streaming technologies including Kinesis and Kafka

• Experience with AWS cloud technologies including S3, Redshift, Spark, Lambda and Kinesis

• Extensive knowledge of SQL query design and tuning for performance and accuracy

• Experience with Python, R, or other data-relevant scripting languages preferred

• Experience in an Agile/Sprint working environment preferred

• Knowledge and direct experience using business intelligence reporting tools. (Quicksight, Tableau, Splunk etc.)

• Excellent communication (verbal and written) and interpersonal skills and an ability to effectively communicate with both business and technical teams

• Strong planning and organizing skills to prioritize numerous projects a