Data Engineer

<p>540 is seeking a Data Engineer to support a mission-critical technology modernization effort for the Department of War. You will design, build, and maintain Databricks-based data pipelines and lakehouse capabilities that enable secure data integration, analytics, AI/ML, and operational workloads at enterprise scale.</p> <p>Working with software engineers, AI/ML engineers, cybersecurity teams, and mission stakeholders, you will build scalable and reliable solutions using Databricks, Python, Apache Spark, and Delta Lake. The ideal candidate enjoys solving complex engineering challenges and developing trusted data products that support national defense missions.</p> <p><strong>Location</strong>: Arlington, VA<br><strong>Citizenship &amp; Clearance Requirement</strong>: Per client requirements, candidates must be U.S. Citizens with an active DoW Secret (or higher) clearance<br><strong>Education Requirement:&nbsp;</strong>Bachelor’s degree in Computer Science, Engineering, or a related technical field preferred; equivalent combinations of education and relevant experience will be considered<br><strong>540 Internal Thrive Level: </strong>Data Engineer II or III</p> <p><strong>WHY 540?</strong></p> <p>540 is a forward-thinking company that the government turns to in order to #getshitdone. We don’t just talk about innovation – we deliver it. We break down barriers, build impactful technology, and solve mission-critical problems.</p> <p><strong>HOW YOU’LL DRIVE IMPACT</strong></p> <ul> <li>Design, develop, and maintain Databricks-based data pipelines, data products, and lakehouse capabilities</li> <li>Build automated ETL/ELT pipelines that ingest, transform, and deliver mission-critical data</li> <li>Develop production-grade data-processing solutions using Python, SQL, PySpark, Apache Spark, and Delta Lake</li> <li>Design and maintain data models, schemas, tables, and medallion architecture patterns supporting analytical, operational, and AI/ML workloads</li> <li>Build and operate batch and streaming data pipelines supporting mission requirements</li> <li>Develop and manage Databricks notebooks, jobs, workflows, clusters, and compute resources</li> <li>Implement data-quality checks, automated testing, monitoring, lineage, and metadata-management capabilities</li> <li>Support data discovery, governance, and access controls using Unity Catalog or similar technologies</li> <li>Optimize Spark workloads and Databricks resources for performance, scalability, reliability, and cost efficiency</li> <li>Collaborate with engineers, analysts, and data scientists to deliver reusable data products and mission capabilities</li> <li>Support Databricks deployments using CI