Senior Data Engineer - Cybersecurity

Company Description

For over 30 years, Silicon Valley Bank (SVB) has helped innovative companies and their investors move bold ideas forward, fast. SVB provides targeted banking services to companies of all sizes in innovation centers around the world. 

Job Description

When you work with the world's most innovative companies, you know you're making a difference.

Our clients are the game changers, leaders and investors who fuel the global innovation economy. They're the businesses behind the next medical breakthroughs. And the visionaries whose new technologies could transform the way people live and work.

They come to SVB for our expertise, deep network and nearly forty years of experience in the industries we serve, and to partner with diverse teams of passionate, enterprising SVBers, dedicated to an inclusive approach to helping them grow and succeed at every stage of their business.

Join us at SVB and be part of bringing our clients' world-changing ideas to life. At SVB, we have the opportunity to grow and collectively make an impact by supporting the innovative clients and communities SVB serves. We pride ourselves in having both a diverse client roster and an equally diverse and inclusive organization. And we work diligently to encourage all with different ways of thinking, different ways of working, and especially those traditionally underrepresented in technology and financial services, to apply.

Qualifications

• Essential skills needed include Scripting in Python, SQL, Shell, Yaml (for CICD), AWS Glue/PySpark, AWS Athena, AWS Lambda/Python, AWS Lakeformation

• Advantageous skills would include Databricks, EMR, & Big Data technologies

• Advanced understanding of both SQL and NoSQL technologies

• Understand and implement secure coding practices

• 6+ years as a Python, PySpark, SQL developer; building scalable ETL applications and data warehouses

• Advanced proficiency programming in Python & PySpark ETL modules is required

• Experience in working with and processing large data sets in a time-sensitive environment while minimizing errors

• Hands-on experience working on On-premise and Cloud data processing/movement solutions.

• Hands-on experience working with big data technologies (Hadoop, Hive, Spark)

• Proficient experience working within the AWS and AWS tools (S3, Glue, EMR, Athena, etc)

• Experienced in maintaining infrastructure as code using Terraform or cloud formation

• Experienced in building Data Visualizations using automations. Hands-on experience working with Tableau and BI tools

• Solid understanding of data warehouse design patterns and best practices

• Ability to develop test plans and stress test platforms

• Experience with complex Job scheduling

• Effective analytical, conceptual, and problem-solving skills

• Must be organized, disciplined, and task/goal oriented

• Able to prioritize and coordinate work through interpretation of high-level goals and strategy

• Effective team player with a positive attitude

• Strong oral and written English language communications skills

Responsibilities

• The Data Engineer is responsible for operationalizing data pipelines that support metrics & analytics initiatives for the company.

• The primary responsibilities include designing, building, managing, optimizing and documenting data flows from various sources into our enterprise data lake

• Delivery of high-quality data is a key item of focus

• The data engineer is expected to collaborate with data scientists, data analysts and other data consumers to productionize data models and algorithms developed by those users to improve the overall efficiency of advanced analysis projects

• Additionally, the data engineer is responsible for ensuring data quality, governance and data security procedures are met while curating data for use in the Data Lake

• Design and develop Lambda and AWS Batch scripts in Python

• Design and incorporate error handling & Data Quality processes into pipelines and processes

• Design, implement, and analyze robust test plans and stress tests

• End-to-end Implementation and monitoring of Data Pipelines

• Lead and/or work with cross-disciplinary teams to understand, document and analyze customer needs