Principal Platform Engineer – Data Ops Engineer

DIRECTV Data Analytics and Operations Team is looking for a curious, talented highly motivated Data Ops Engineer to lead the automation, orchestration, and optimization of our cloud-based data workflows across Snowflake, Databricks, and AWS. The ideal candidate will have deep expertise in Apache Airflow, CI/CD, automation, and performance monitoring, with a passion for building scalable, efficient, and high-performance data operations solutions.

In this role, you will work closely with data engineering, DevOps, security, and business teams to design and implement next-generation orchestration, automation, and data deployment strategies that drive efficiency, reliability, and cost-effectiveness.

This is the perfect opportunity to become a part of a fast paced and innovative team that solves real world problems and drive business value.

Key Responsibilities

1. Data Orchestration & Workflow Management (30%)

• Design, implement, and maintain workflows using Apache Airflow to automate data pipeline execution.
• Optimize orchestration logic, manage task dependencies, retries, and error handling to ensure reliable data operations.
• Integrate workflow orchestration across Snowflake, Databricks, and AWS environments.

2. Programming & Automation (25%)

• Develop and enhance automation scripts using Python, Spark, JavaScript, and Shell scripting to streamline data workflows and reduce manual intervention.
• Implement system-level automation using orchestration and scripting frameworks to optimize data processing efficiency.

3. CI/CD Integration & Deployment (20%)

• Design, build, and maintain CI/CD pipelines using Jenkins, GitLab, and AWS CodePipeline for automated testing, deployment, and workflow execution.
• Automate infrastructure provisioning, workflow deployments, and monitoring using Infrastructure as Code (IaC) methodologies.
• Ensure smooth and efficient integration of data pipelines into existing CI/CD processes to enhance agility and scalability.

4. Performance Monitoring & Optimization (15%)

• Utilize monitoring tools such as SonarQube, JMeter, Datadog, and AWS CloudWatch to track, test, and optimize data pipeline performance.
• Conduct performance tuning and scalability assessments for Snowflake, Databricks, and orchestration workflows.
• Proactively identify and resolve system bottlenecks to enhance reliability and efficiency.

5. Incident Management & Collaboration (10%)

• Provide Tier 2/3 support for complex data workflow issues, conducting root cause analysis and implementing preventive measures.
• Work cross-functionally with Data Engineering, DevOps, and Security teams to troubleshoot and resolve operational challenges.
• Develop runbooks and incident resolution playbooks to improve troubleshooting efficiency.

Experience Requirements

• 5+ years of experience in data operations, orchestration, and automation, with a proven track record of managing large-scale data workflows.
• 5+ years of hands-on programming experience with Python, Spark, JavaScript, and Shell scripting for data automation.
• 5+ years of experience in CI/CD pipeline development using Jenkins, GitLab, or AWS CodePipeline.
• 5+ years of expertise in Snowflake, Databricks, and AWS, focusing on workflow orchestration and performance optimization.
• 5+ years of experience with performance monitoring using SonarQube, JMeter, Datadog, or CloudWatch.
• 5+ years in Tier 2/3 support roles, specializing in incident resolution, troubleshooting, and workflow optimization.
• 4+ years of experience with dbt.
• Deep expertise in Apache Airflow and orchestration technologies to manage complex data workflows.
• Strong analytical and problem-solving skills to diagnose and resolve complex data pipeline issues.

Preferred Certifications

• Databricks Certified Data Engineer Associate / Professional
• SnowPro Advanced Architect Certification
• AWS Certified DevOps Engineer – Professional
• Apache Airflow Certification
• ITIL 4 Managing Professional (for incident management expertise)
• Certified ScrumMaster (CSM) (for agile collaboration skills)

Education

• Bachelor’s or Master’s degree in Computer Science, Data Engineering, or a related field preferred.

May require a background check due to job duties requiring routine access to DIRECTV and DIRECTV customer’s proprietary data. Qualified applicants with arrest and conviction will be considered for employment in accordance with local ordinances and state law.

This is a remote position that can be located anywhere in the United States, with preferenc