Sr. Software Engineer

About Blackhawk Network

Today, through BHN’s single global platform, businesses of all kinds can tap into the world’s largest network of branded payment solutions. BHN helps businesses grow revenue, increase loyalty, motivate and reward their teams, disburse funds and engage consumers. Branded payment solutions include the issuance and distribution of gift cards, egifts, corporate payouts and rewards, along with the technology to deliver these products in seamless, integrated ways. BHN’s network spans the globe with more than 400,000 consumer touchpoints. Learn more at BHN.com.

 

Hybrid flexibility: At Blackhawk Network, you’ll enjoy the best of both worlds—focused remote work plus in-person collaboration at our Coppell, TX office. This rhythm gives you the tools, connection, and autonomy you need to make a real impact.

Overview

We are looking for a Senior or Staff Software Engineer with deep Site Reliability Engineering expertise and strong Python development skills to join our SRE team. You will own the reliability, scalability, and observability of a critical Data pipeline platform.

 

This is a builder and operator role. you will write production-grade Python services, and implement observability-as-code infrastructure, and drive reliability improvements across a complex AWS-native platform. You will work at the intersection of software engineering and infrastructure, eliminating toil through automation, building internal developer platforms, and ensuring our systems meet the reliability bar our partners and customers expect.

Responsibilities

Site Reliability Engineering

• Own and improve service reliability, availability, and performance across a distributed AWS platform (EKS, EC2, RDS, S3)
• Define and track SLOs, SLIs, and error budgets for critical services and partner-facing integrations
• Drive initiatives identify manual, repetitive operational work and eliminate it through automation
• Partner with engineering teams to embed reliability practices into the SDLC

Platform & Observability Engineering

• Build and maintain observability-as-code infrastructure using Terraform and New Relic NerdGraph 
• Design multi-signal observability pipelines metrics, logs, traces  across Splunk, New Relic, and OpenSearch
• Architect and maintain log routing and sawmills data pipelines using FluentD and Sawmills
• Build New Relic NerdGraph integrations and custom NRQL dashboards for platform health visibility
• Maintain and extend Terraform modules for AWS infrastructure provisioning  Security Groups, EKS node groups, Aurora, IAM roles
• Contribute to the MCP server architecture (Python-based internal developer tooling) integrating Jira, Confluence, Bitbucket, and observability platforms

Python Development

• Design, build, and maintain production-grade Python services and automation REST APIs, background workers, CLI tooling, and data pipeline adapters
• Mentor junior and mid-level engineers  code reviews, pair programming, architecture guidance
• Work closely with program managers and product stakeholders 

 

 

 

Qualifications

• Bachelor's degree in Information Technology, Computer Science, or related field; or equivalent experience.
• 7+ years of software engineering experience 
• 4+ years of SRE or platform engineering experience in a production environment
• Strong Python proficiency production services, automation, REST APIs 
• Experience with AWS EKS, EC2, RDS/Aurora, S3, IAM, VPC, CloudWatch, Security Groups
• Hands-on Terraform experience  writing modules, managing state, CI/CD integration
• Experience with observability platforms  Splunk New Relic (NRQL, NerdGraph)
• Strong understanding of SRE principles SLOs/SLIs/error budgets, toil reduction, incident management, capacity planning
• Experience with Kubernetes / EKS  pod operations
• Strong Linux/ Unix fundamentals log management, performance debugging

Preferred Experience

 

• Experience with log pipeline tooling  FluentD, Open Telemetry
• Familiarity with Splunk, New Relic, OpenSearch, Elasticsearch for log storage and search
• Experience building internal developer platforms or tooling  CLI tools, internal APIs, automation frameworks
• Familiarity with MCP (Model Co