Sr. Cloud Data Engineer

Company

Federal Reserve Bank of San Francisco

We are the Federal Reserve Bank of San Francisco—public servants with a mission to advance the nation’s monetary, financial, and payment systems to build a stronger economy for all Americans. We are a community-engaged bank, and are committed to understanding and serving the vibrant, expansive communities of the Twelfth District. We respect people for what they do and for who they are. We build opportunities to learn and grow. We empower our people to balance their life and work responsibilities. That’s why we offer a flexible hybrid work model that allows you to collaborate with office colleagues on some days, and work from home on others.

We are looking for a Sr Cloud Data Engineer to join the Data and Analytics Service (DAS) Team. Our team delivers a centralized data and analytics platform that provides advanced data migration and analytics solutions for our customers for data storage, data processing, data integration, data lakes, data warehousing, and data analytics. You will work cross-functionally with Data Architects, Product Owners, Design Engineers, Developers and Business Analysts in a rapidly evolving environment, identifying solutions, building, and implementing Infrastructure as Code using our CI/CD pipeline. You will have an opportunity to showcase your critical thinking and technical skills across many disciplines. This role will be instrumental in shaping the development and support of our product roadmaps and will advance our cloud data analytics platforms.

Essential Responsibilities

• Understand technology vision and strategic direction of business needs

• Participate in defining technical requirements that outlines solutions for enabling data analytics within an AWS cloud environment build on data mesh architecture

• Design integrated solution in alignment with design patterns, blueprints, guidelines, and standard methodologies for data platform products

• Participate in developing architecture and design conceptual solutions by incorporating cloud native and 3rd party vendor products

• Provide analysis of current state, future state, and partner with the Product manager to define key outcomes for an IT data platform strategy

• Participate in research and perform POCs (proofs of concept) with emerging technologies and adopt industry best practices in the data space for advancing the cloud data platform using data mesh architecture

• Design, develop, and implement Low-Code ETL solution

• Developing RESTful APIs and API Gateway integration

• Develop automation through Infrastructure as a code to increase build efficiency in the cloud using Terraform, Python, and OS shell scripts

• Develop scripting to integrate multiple data sources to data and analytics platform

• Build platform supporting HA and disaster recovery

• Develop data streaming, migration, and replication solutions

• Provide consulting and guidance on business solution development in the data and analytics space and cloud technologies

• Demonstrate leadership, collaboration, exceptional communication, negotiation, strategic and influencing skills to gain consensus and produce the best solutions

• Engage with Senior leadership, business leaders at the Federal Reserve and the Board to share the business value

• Promote collaboration across business and IT organizations

Qualifications:

• Bachelor’s degree in Computer Science, Management Information Systems, Computer Engineering, or related field or equivalent work experience; advance degree preferred

• 7+ years in designing and building ETL services with all aspects of cloud design, development, and implementation

• 5+ years of experience with full stack development, including application, API, testing, and automation experience using at least one of the following languages (Terraform, JSON, Python)

• 3+ years of experience designing, building, and supporting large scale solutions hosted either on-prem or in the cloud (AWS), and experience with migrating data to the cloud

• Experience with maximising CI/CD pipelines and GitLab is a must

• Hands on experience in Terraform, Python, and OS shell scripts including automation tools

• Hands on experience in one or more languages such as Java, JavaScript

• Experience in one of the following Immuta, Starburst or Collibra workflow is preferred

• Experience with other AWS data related services including various compute and storage offerings

• Experience with RDS, NOSQL Databases, RDBMS and in-memory databases

• Must be familiar with Agile methodologies and able to work in an