Lead Data Engineering

Job Overview

We seek an experienced and innovative, hands on Senior Architect and Tech Lead to design, optimize, and implement a cutting-edge data architecture in collaboration with our Business and Client teams. You will leverage your expertise in developing, enhancing, and managing large-scale data systems to drive strategic initiatives and elevate the capabilities of our data platforms.

This role will spearhead the strategic planning, organizational management, and execution of our data engineering roadmap, collaborating with cross-functional teams to deliver a unified customer view and enable data-driven decision-making.

Your primary responsibility will be to create an enterprise-wide data strategy and architecture that streamlines the entire data lifecycle - from ingestion to retrieval - ensuring seamless integration, scalability, and high-quality data throughout. You will maintain rigorous standards for security, governance, and data quality to ensure trustworthiness and compliance.

You will partner with cross-functional groups to transform our existing data infrastructure into a modern, cloud-native solution that fuels business insights and drives value. Your focus will include migrating core business processes across diverse systems and locations to a unified data layer, enabling technical teams to publish data and end-users to access it seamlessly.

To achieve these goals, you will be asked to develop tooling and services that connect data sources to consumers, guaranteeing accessible, reliable, and trustworthy data delivery that meets business needs. If you're passionate about crafting robust data architectures and driving innovation through technology, we encourage you to apply for this exciting opportunity. 

Responsibilities:

• Design, develop, and maintain data systems, analytics, and reporting applications

• Engage with our multiple business teams to understand their needs and requirements and develop cohesive strategies to address them in a cost-effective and efficient manner

• Assess current environment and make recommendation specific to data and reporting technologies

• Collaborate with development teams to ensure the functionality being developed is implemented in a platform-centric manner, while setting data for publishing new data sources

• Build reporting capabilities that allow for self-service analysis based on a multiple “what if” scenarios leveraging AI technologies

• Optimize existing data systems and analytics applications

• Analyze data to identify trends and develop insights

• Prioritize appropriately so our existing data estate continues to work flawlessly as the target architecture is developed

• Implement processes to ensure the quality and availability of data and build functionality to improve the overall robustness of our data platform

• Develop accurate metrics to identify performance improvement opportunities

• Design verification plans and implement unit tests to validate the correctness and performance of the solutions

• Design a reliable production support process and, when required, assist in supporting and troubleshooting production issues

• Foster strong partnerships with technology vendors

• Negotiate and manage technical contracts to support the organization's needs and ensure cost-effective solutions

• Automate all aspects of the data lifecycle

• Stay updated with industry trends and advancements in data engineering,
  evaluating and recommending suitable technologies and tools.

Qualifications:

• At least 10 years of experience working with modern, large-scale data systems in a technical leadership role

• Possess extensive expertise in working with diverse data storage technologies, including databases, data warehouses  data lakes and data lakehouses

• Proven track record of building and scaling data platforms at an
  enterprise level.

• Excellent software architecture design skills

• Exceptional communication skills and the ability to build strong relationships as well as credibility

• Excellent leadership and management skills, with the ability to mentor and develop high-performing teams. Expertise in budget control and vendor management
  preferred.

• Strong strategic planning and execution abilities, with a focus on aligning data

• Bachelor’s degree in Computer Science, Computer Engineering or a related discipline

• Experience with containerized deployments and elastic environments

• Thorough knowledge of with CI/CD (Jenkins, Gitlab, Github) and testing (e.g., pytest) methodologies

• At least 3 years of experience