Senior Observability Engineer

Responsibilities

Peraton is seeking a Senior Observability Engineer to join our team of highly skilled, dedicated, and motivated cleared professionals supporting the Department of State (DoS) Consular Affairs Enterprise Infrastructure Operations (CAEIO) Program, for the Bureau of Consular Affairs (CA). The program provides IT Operations and Maintenance to modernize the legacy networks, applications, and databases supporting CA services globally.

 

Responsibilities

• Perform full lifecycle engineering activities (design, develop, test, implement, maintain) for new and existing observability tools/platforms in an enterprise environment with on-premises and cloud-integrated systems/applications/infrastructure.
• In support of Observability Platform proof-of-concepts, pilots, and production implementations: Gather functional/business requirements, establish success criteria, and develop use cases.
• Proactively design telemetry strategies to gain real-time insights and identify potential issues before they escalate.
• Apply knowledge and experience to the following:
  • Telemetry data collection, analysis, and implementation to derive meaningful insights from different sources including metrics, events, logs, and traces.
  • Distributed systems including those with microservices and hybrid infrastructure (cloud/on-premises) to effectively design telemetry pipelines, build monitoring systems, and implement observability practices.
• Identify patterns, detect anomalies, and build a holistic understanding of system/application/infrastructure behavior.
• Collaborate across different observability domains to include Infrastructure, Applications (APM), Networking and close these gaps with the cross-functional skills.
• Provide technical leadership and support to the following: detecting anomalies, troubleshooting incidents, monitoring system health, optimizing resource allocation, enhancing user experience, supporting compliance and security effort.
• In accordance with scaled agile framework (SAFe), leverage tools (Jazz/Jira) to track work and backlog.
• Work with Team Lead and/or Observability Project Manager to prioritize efforts and meet deliverable timelines.
• Assist in development of observability backup recovery methodologies.
• Design/develop/deploy capabilities that comply with security/integrity controls and limit operational downtime for maintenance activities.
• Participate in briefing program leadership, the government customer, and other stakeholders.
• Leverage approved systems for incident/change management (SNOW), work items (Jazz/Jira), documentation (Confluence), and others.
• Support adhoc data calls from the government customer in support of observability systems.

 

Core Work Schedule: First Shift, approximately 7:00 am – 3:30 pm, Monday–Friday.

 

Work Location: This position is remote. However, the candidate must be local to the DMV area. Employees can work in-person with other team members in Sterling, VA on Wednesdays (location is 5 minutes from Dulles Airport).

Qualifications

Required Qualifications

• Active SECRET clearance with ability to obtain Top Secret.
• Five plus (5+) years observability platform/tools experience.
• Subject matter expertise in telemetry.
• Familiarity with at least one of the .NET languages (e.g. C#, Visual Basic .NET) and HTML5/CSS3.
• Familiarity with architecture styles/APIs (REST, RPC).
• Familiarity with Java programming and/or code debugging/testing.
• Familiarity with software development lifecycle.
• Experience with underlying databases used by observability platforms.
• Familiarity with the ASP.NET framework, SQL Server and design/architectural patterns (e.g. Model-View-Controller (MVC)).

 

Desired Qualifications

• Expertise in one or more of the following Observability platforms: Cisco AppDynamics, Datadog, Dynatrace, Splunk, others.
• COTS Application Development.
• SQL, Oracle DB, Apache/IIS, RHEL/OL.
• Bachelor’s degree in STEM field.
• Observability Platform certifications.

 

Education and Experience: Minimum of 16 years with HS diploma. Minimum of 12 years with BS/BA; Minimum of 10 years with MS/MA; Minimum of 7 years with Ph.D.

Peraton Overview

Peraton is a next-generation national security company that drives missions of consequence spanning the globe and extending to the farthest reaches of the galaxy. As the world’s leading mission capability integrator and transformative enterprise IT provider, we deliver trust