Data Engineer II

It's exciting to work for a company that makes the world measurably better.

We're committed to bringing safety, quality, and customer focus to the business of advanced ceramics manufacturing.

Job Title

Data Engineer II

The Data Engineer II plays a critical role in developing, transitioning and optimizing data transformations that enable efficient data processing and analysis. Collaboration with internal and cross-functional teams will drive data-driven decision-making and contribute to the continuous improvement of our Lakehouse hosted on databricks.

Roles & Responsibilities:

• Work with large datasets, ensure data quality, accuracy, and performance.
• Follow User Story requirements and acceptance criteria to deliver outcomes as defined by senior team members.
• Leverage databricks SQL to curate and transform data as defined by project plan or user stories.
• Follow data transformation, integration, and validation guidelines to support analytics and reporting needs.
• Optimize and fine-tune data pipelines for improved speed, reliability, and efficiency.
• Troubleshoot and resolve data-related issues, collaborating with the team to identify root causes.
• Document data processes, data lineage, and technical specifications for future reference.
• Participate in code reviews, ensuring adherence to coding standards and best practices.
• Collaborate with DevOps teams to automate deployment and monitoring of data pipelines.

Job Requirements:

Education:

Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

Experience:

• Proven experience in data engineering, with a strong track record of following technical requirements and implementing data solutions.
• Proficiency in programming languages such as SQL, Python, or Scala, and experience with Spark data processing frameworks.
• Strong knowledge of SQL warehouse and Lakehouse databases.
• Experience in data product development, analytical models, and model governance.
• Experience in anonymizing data and managing the use of data.
• Experience in data hygiene procedures, and data management.
• Familiarity with industry best practices for collection and use of data.
• Outstanding problem-solving skills and the ability to navigate complex MSFT data environments.
• Effective communication and collaboration skills to work with both technical and non-technical stakeholders.
• Experience with agile methodologies and DevOps practices.

Functional/Technical Knowledge, Skills & Abilities:

• Databricks data engineering persona experience.
• dbx/ANSI SQL experience.
• Ability to dissect SSIS packages, stored procedures, and views.
• Passionate about data and the impact of data analytics on an organization.
• Solid understanding of multiple business domains, especially manufacturing.
• Ability