Lead Associate Principal, Software Engineering: Software Development Test (SDET)

To be considered for this position, applications and resumes are accepted only through our careers site by directly applying to the posted job. We do not accept unsolicited resumes or sales solicitations from staffing agencies. Any OCC employee wishing to submit a referral must do so through their Workday account. Any resume submitted outside of an active job posting will not be considered for employment.

What You'll Do:

OCC's Enterprise Quality Engineering organization seeks an Associate Principal in Performance Testing to design and execute comprehensive performance and failover testing strategies across mission-critical clearing, risk, and settlement systems. This role combines hands-on performance engineering expertise to ensure systems meet regulatory RTO/RPO requirements, handling multi-region failover scenarios, cloud-native architectures, and distributed event-driven systems.

Primary Duties and Responsibilities:

To perform this job successfully, an individual must be able to perform each primary duty satisfactorily.

• Define test plans, load profiles, and ramp scenarios for high-volume transaction systems

• Execute tests using LoadRunner, JMeter, and Gatling; analyze results and recommend optimizations

• Lead DR testing across primary, secondary, and tertiary regions

• Build and maintain performance test automation using Python, Java, and open-source tools

• Develop CI/CD-integrated pipelines (Jenkins, GitHub Actions) for continuous performance validation

• Implement regression testing to catch performance degradation from code changes

• Validate application behavior across multi-region deployments

• Define and validate RTO/RPO targets for critical applications

• Design failover scenarios (region switching, data replication, automation, post-failover validation)

• Partner with infrastructure/application teams to verify state consistency and data integrity during failover

• Test replication lag, failover triggers, and cross-region data consistency

• Validate Kafka producer/consumer behavior under failover conditions

• Confirm performance SLAs hold during region-switching scenarios

• Integrate test results with observability platforms

• Maintain and evolve LoadRunner scripts and environments

• Document methodologies, baselines, and trend analysis

• Mentor QE engineers on performance and failover testing techniques

• Collaborate with development, DevOps, and infrastructure teams on requirements and capacity planning

• Ensure testing aligns with PGP compliance and financial market infrastructure regulations

• Document test evidence to support regulatory audits

• Validate performance SLAs meet regulatory RTO/RPO requirements

Supervisory Responsibilities:

• None

Qualifications:

The requirements listed are representative of the knowledge, skill, and/or ability required.  Reasonable accommodations may be made to enable individuals with disabilities to perform the primary functions.

• 7+ years hands-on experience in performance testing and engineering, including design and execution of complex performance test scenarios

• 5+ years designing and executing failover/DR testing across multi-region architectures (cloud and hybrid environments)

• Strong expertise in performance testing tools: LoadRunner (enterprise scaling), JMeter, Gatling, with ability to evaluate tool trade-offs

• Proficiency in scripting/programming: Python, Java, JavaScript, with ability to develop performance testing frameworks

• Solid hands-on experience with AWS cloud environments, including EC2, RDS, Kafka, and performance monitoring

• Working knowledge of Kubernetes and containerized application performance testing (Docker, Kafka, event-driven systems)

• Experience with UC4 (Automic) job scheduling in test execution and orchestration

• Hands-on experience testing Kafka producer/consumer performance, lag validation, and message throughput

• Strong understanding of relational databases (SQL Server, Oracle, DB2) and ability to write queries for performance analysis

• Experience with CI/CD pipelines (Jenkins, GitHub, GitLab CI) and integration of performance tests into automation

• Proficiency with APM/observability platforms (Splunk, Dynatrace, AppDynamics) and correlation of test results with production telemetry

• Working knowledge of Linux operating systems and command-line tools for performance analysis

• Experience with t