Principal Data Engineer

Job Summary: 

The Principal Data Engineer is the leading expert member of the Global Data engineering team and is responsible for designing, developing, and maintaining our infrastructure to support reporting and analytics needs across the company. The engineer will work extensively with other engineers, architects, and analysts to provide insights and drive decision making in the Global Data department. The principal engineer should consistently drive best practices of development and have an excellent understanding of the tools used in iHerb's data platform.

Job Expectations:

• Designs and develops pipelines that support data ingestion, curation, and provisioning of complex enterprise data to support analytics and reporting in our current technology stack.

• Provides successful deployment and provisioning of data solutions to required environments.

• Designs and builds data architecture and applications that successfully enable speed, quality, and efficient pipelines.

• Responsible for the data pipeline continuous integration and continuous delivery (CI/CD) processes.

• Manages data pipeline jobs throughout their lifecycle.

• Assist in the design and build efficient data models for robust business intelligence, analytics, and engineering needs that remain.

• Demonstrate initiative by seeking potential business issues and proactively solve them.

• Analyze and translate business needs into data models to support long-term, scalable, and reliable solutions.

• Interacts with cross-functional customers and development team to gather and define requirements.

• Reviews discrepancies in requirements and resolves with stakeholders in a timely manner.

• Build strong cross-functional partnerships with Data Scientists, Analysts, Product Managers and Software Engineers to understand data needs and deliver on those needs.

• Continuously improves understanding of the data and applications across the business.

• Lead processes that ensure site reliability for our data stack. 

• Optimize and tune code performance. 

• Develop best practices for standard naming conventions and coding practices to ensure consistency of data models and tracking. 

• Actively engages with other technical teams to make recommendations on cohesive infrastructure guidelines.

• Champion the use of the latest innovations.

• Partner with IT and Legal to design secure and automated processes and implement practices that enable data democracy and agility.

• Identifies and recommends appropriate data quality validations and ensures integrations are automated and have proper exception handling. 

• Leads pipeline code and metadata framework changes. 

• Engages with other development teams upstream to proactively understand downstream impacts 

• Actively pursues industry developments and makes suggestions on best practices across the architecture. 

• Run, guide, and implement database administration responsibilities and continuously automate relevant processes

• Leads pipeline code and metadata framework changes.

• Seeks out opportunities to elevate fellow engineers’ abilities and experience and mentor them to upgrade their skills.

The duties and responsibilities described above may provide only a partial description of this position. This is not an exhaustive list of all aspects of the job.  Other duties and responsibilities not outlined in this document may be added as necessary or desirable, with or without notice.

Knowledge, Skills and Abilities:

Required:         

• 7+ years of programming skills with Python.

• 3+ years of experience working with API’s.

• Have experience with Docker and/or Kubernetes.

• Proven experience working with large datasets.

• Proficient with shell scripting.

• Proficient in building automated testing within CICD.

• Experienced in Agile methodologies & DevOps approach to maintaining pipelines and databases.

• Excellent knowledge of software engineering fundamentals.

• Deep understanding of data lifecycles, data computation principles, data stores and a solid understanding of CICD principles.

• Proficiency with Databricks (DLT, Medallion Architecture, Lakehouse Concepts, etc)

• Proven experience building scalable data platforms professionally.

• Approves d