Senior Python Data Engineer

<p>We are looking for a <strong>Senior Python Data Engineer</strong> to join an innovative AI-driven migration program aimed at modernizing legacy data processing applications through intelligent automation.</p><p>In this role, you will combine strong software engineering expertise with modern AI-assisted development practices to help industrialize an AI-powered migration factory. You will actively contribute to the development of high-quality Python data pipelines while improving the orchestration of AI agents responsible for automating large parts of the migration lifecycle.</p><p>This position is ideal for someone who enjoys solving complex data engineering challenges while shaping the future of AI-assisted software development</p><p><strong>Key Responsibilities</strong></p><p><strong>Python Data Engineering</strong></p><ul><li>Design, develop, and optimize Python-based data transformation solutions.</li><li>Review, validate, and improve AI-generated Python code to ensure production readiness.</li><li>Ensure high standards of code quality, performance, maintainability, and operational reliability.</li><li>Support complex migration scenarios requiring manual refinement beyond automated code generation.</li><li>Develop and optimize modern data processing pipelines.</li><li>Contribute to data quality, schema evolution, metadata management, and Parquet optimization.</li><li>Support distributed data processing using PySpark when appropriate.</li></ul><p><strong>AI-Assisted Development &amp; Agent Engineering</strong></p><ul><li>Act as a key contributor to an AI-assisted software migration platform.</li><li>Improve the orchestration framework responsible for coordinating multiple specialized AI agents.</li><li>Enhance collaboration between AI agents responsible for:&nbsp;<ul><li>Legacy SAS code analysis</li><li>Functional analysis generation</li><li>Technical design generation</li><li>Python code generation</li><li>Automated validation and testing</li></ul></li><li>Improve workflow orchestration, traceability, scalability, and reliability.</li><li>Strengthen human-in-the-loop validation processes and feedback mechanisms.</li><li>Identify opportunities to increase automation while maintaining software quality and governance.</li><li>Contribute to the continuous evolution of AI-assisted development practices and engineering standards.</li></ul> <p><strong>Required Skills &amp; Experience</strong></p><p><strong>Python &amp; Data Engineering</strong></p><ul><li>Minimum <strong>5 years of professional experience</strong> developing Python applications.</li><li>Strong expertise in:&nbsp;<ul><li>Python</li><li>Pandas</li><li>Polars</li><li>PyArrow</li><li>Parquet</li><li>Unit and integration testing frameworks</li></ul></li><li>Proven experience building and maintaining:&nbsp;<ul><li>Data engineering solutions</li><li>ETL / ELT pipelines</li><li>Data transformation workflows</li><li>Data quality controls</li><li>Schema management</li></ul></li><li>Experience with <strong>PySpark</strong> is considered a strong advantage.</li></ul><p><strong>AI-Assisted Development</strong></p><p>Experience working with modern AI-assisted software engineering tools and workflows, including:</p><ul><li>GitHub Copilot or equivalent AI coding assistants</li><li>AI-assisted software development</li><li>Multi-agent systems</li><li>Agentic workflows</li><li>Agent orchestration frameworks</li><li>Human-in-the-loop validation processes</li><li>Workflow automation and optimization</li></ul>