Senior Data Engineering Lead

BOOST LLC is a dynamic management consulting firm that offers an array of government-compliant back-office solutions to support our teaming partners within the GovCon space. Our consultants are experts in the areas of Accounting, Contracts, Human Resources, Recruiting & Sourcing, and Strategic Pricing and our passion is to guide and propel our partners towards success within this competitive sector.

BOOST is partnering with a cleared small business that provides mission-critical, AI-driven cyber defense and reverse engineering solutions for U.S. Government clients to hire a Senior Data Engineering Lead. This opportunity is 100% onsite located in the Northern Virginia area.

Position Summary:

As the Senior Data Engineering Lead, you will be the senior technical authority driving the design, implementation, and sustainment of high-performance ETL pipelines, data normalization frameworks, and schema standardization strategies across multiple secure enclaves. Your work will directly power mission-critical AI/ML analytics, hunt operations, and executive decision dashboards. This role demands absolute commitment to mission-first, people-always values, enforcing data-governance policies and mentoring engineers with zero tolerance for mission failure. You will serve as a trusted steward of data architecture supporting decisive mission execution at the highest security and performance levels.

Responsibilities:

• Architect scalable, fault-tolerant ETL pipelines using Spark, NiFi, Kafka, and Python to ingest and transform petabyte-scale telemetry under secure mission conditions.

• Design and maintain canonical schemas and robust data normalization layers (Parquet, Delta, Iceberg) to enable consistent, query-ready, high-value datasets.

• Develop automated data-quality checks, data lineage tracking, and anomaly-detection alerting systems, fully integrated with data-governance catalogs.

• Collaborate with AI/ML engineers, hunt analysts, and DevSecOps teams to optimize data partitioning, serialization formats, and feature engineering pipelines for training and real-time analytics.

• Implement granular security controls including row-level, column-level, and field-level encryption while ensuring compliance with NIST standards and classified data-handling protocols.

• Maintain and optimize performance dashboards, cost dashboards, cluster sizing, storage tiering, and caching strategies to deliver best-in-class mission performance.

• Lead code reviews, enforce CI/CD practices for data pipeline deployments, and mentor junior data engineers to build a high-performing, secure engineering culture.

• Author and maintain architectural documentation, data dictionaries, and SOPs; deliver technical briefings to mission leadership and program stakeholders.

Required Qualifications:

• Eight plus (8+) years in data engineering or large-scale analytics platform development in classified, highly regulated, or mission-critical environments.

• Expertise with distributed processing frameworks (Apache Spark, Flink), streaming platforms (Kafka, Kinesis), and relational/NoSQL data stores.

• Proven experience designing canonical data models, normalization pipelines, and schema-evolution strategies.

• Proficiency in Python for data engineering tasks and automation.

• <