Senior Data Engineer

Abbott is a global healthcare leader that helps people live more fully at all stages of life. Our portfolio of life-changing technologies spans the spectrum of healthcare, with leading businesses and products in diagnostics, medical devices, nutritionals and branded generic medicines. Our 114,000 colleagues serve people in more than 160 countries.

     

JOB DESCRIPTION:

For years, Abbott’s medical device businesses have offered technologies that are faster, more effective, and less invasive. Whether it’s glucose monitoring systems, innovative therapies for treating heart disease, or products that help people with chronic pain or movement disorders, our medical device technologies are designed to help people live their lives better and healthier. Every day, our technologies help more than 10,000 people have healthier hearts, improve quality of life for thousands of people living with chronic pain and movement disorders, and liberate more than 500,000 people with diabetes from routine fingersticks. 

Our location in Sylmar, CA, or Santa Clara, CA, currently has an opportunity for a Senior Data Engineer within the Global Data Science & Analytics organization. The Senior Data Engineer will be responsible for designing, building, and maintaining robust data pipelines using Azure Databricks and PySpark. He or she will develop and optimize ETL pipelines, leverage Apache Spark for data processing, and collaborate with data scientists, clinical scientists,  and analysts to deliver tailored solutions. The candidate will craft complex SQL queries, apply python/R for advanced data manipulation, and implement stringent data governance practices. The Data Engineer will manage multiple projects concurrently, establish best practices for data quality and security, and troubleshoot data-related issues promptly. He or she will stay abreast of the latest trends in data engineering and communicate effectively with team members and stakeholders. 

Responsibilities 

• Data Pipeline Development: Design, build, and maintain robust data pipelines using Azure Databricks and PySpark, ensuring optimal performance and scalability. 

• ETL Implementation: Develop and optimize ETL pipelines utilizing Databricks tools to facilitate efficient data processing and transformation. 

• Data Processing: Leverage Apache Spark for both batch and incremental data processing, ensuring data integrity and consistency. 

• Collaboration: Work closely with data scientists and analysts to comprehend data requirements and deliver tailored solutions that meet business needs. 

• SQL Development: Craft and refine complex SQL queries for effective data extraction, analysis, and reporting. 

• R Utilization: Apply R for advanced data manipulation and statistical analysis, enhancing data insights (nice to have). 

• Data Governance: Implement stringent data governance practices using tools such as Unity Catalog to ensure data quality, security, and compliance. 

• Cloud Integration: Integrate Databricks with cloud platforms, with a preference for Azure, to streamline data workflows and enhance accessibility. 

• Project Management: Manage multiple projects simultaneously, ensuring timely delivery and adherence to high-quality standards. 

• Be