Data Engineering Lead, Vice President

Who we are looking for

Enterprise Data organization is recruiting a Data Governance Technical Lead to help drive technical implementation and management for the Collibra Data Intelligence platform implementation. Technical lead will collaborate with business unit data governance leads and solutions team to rollout the functional capabilities for the data governance platform.

We are looking for a Data engineer for Vendor SaaS Cloud application. The candidate should have industry experience (preferably in Financial Services) in supporting enterprise applications and exposure to Cloud based Data engineering platforms with deep understanding of data processing frameworks and solid experience with ETL development using distributed computing frameworks as Apache Spark, Hadoop, Hive. This hands-on role will partner with senior-level development managers, architects and business leadership to develop and execute the technology product roadmap.
 

What you will be responsible for

•     End to end Cloud Data Lake design & development including data ingestion, data modeling and data distribution.

•     Build data integrations, hand-offs between on-prem/cloud-based systems.

•     Design and develop high volume data ingestion components and solutions to collect data from various sources.

•     Manage and use appropriate tools in the cloud data lake development and deployment.

•     Responsible for developing/implementing cloud infrastructure to support current and future business needs.

•     Participating in technial design, application build, configuration, unit testing, and production deployment.

•     Ensuring all cloud solutions exhibit higher level of cost efficiency, performance, security, scalability, and reliability

•     Management oversight of a global team of software development and QA engineers.

•     Building strong partnership with technology & business leaders and peers

•     Work closely with the Product Owner and others on requirements analysis, planning, project management, and execution.

What we value

These skills will help you succeed in this role

• Development experience in designing, developing and deploying complex applications.

• Strong skill-set, experience in PL/SQL, Java/J2EE, Python, Scala/PySpark, PERL/scripting .

• Hands on experience with Cloud platforms, preferably using Databricks.

• Experience in building systems using modern, scalable, resilient, cloud native architectures.

• Experience in Data Modeling and database design with strong skills in performance tuning.

• Hands on experience in setting up data pipelines for ELT jobs across multiple data sources.

• Hands on experience in big-data processing, troubleshooting and resolving database performance issues.

• Strong knowledge of Cloud Security in AWS/Azure, specifically IAM Roles & Polices, Security Groups and Encryption keys.

• Experience with optimizing database performance, scalability, ensuring data security and compliance.

• Experience with event based, micro-batch and batched high volume and high velocity transaction and data processing system.

• Proficiency with CICD and DevOps.

Education & Preferred Qualifications

• Bachelor’s/Master's Degree in a Computer Science, Engineering or Math or equivalent experience

• 10+ years' experience in professional software design and development.

• Strong analytical skills, with ability to collect, absorb, and analyze information quickly and independently

• Proven ability to develop and work with cross functional processes and project timelines

• Excellent follow-through, attention to detail and time management skills                                               

• Adapt and excel at managing multiple simultaneous tasks in a high pressure, deadline environment.

Additional requirements

Are you the right candidate? Yes!

We truly believe in the power that comes from the diverse backgrounds and experiences our employees bring with them. Although each vacancy details what we are looking for, we don’t necessarily need you to fulfil all of them when applying. If you like change and innovation, seek to see the bigger picture, make data driven decisions and are a good team player, you could be a great fit.

Why this role is important to us

Our technology function, Global Technology Services (GTS), is vital to State Street and is the key enabler for our