Senior Data Engineer (Data migration)

<p>We're looking for an experienced Data Engineer for a Swedish multinational telecommunications company and mobile network.</p><p>As a Senior Data Engineer, you will play a pivotal role in migrating our data from Hadoop to a new on-premises open-source tools-based data platform.</p><p>This role requires deep data engineering expertise, focusing on Spark, Scala, Airflow, Hadoop, and Python.</p><p></p><h3>Responsibilities</h3><ul> <li>Lead the migration of data from Hadoop to a new on-premises open-source tools-based data platform.</li> <li>Design and implement robust data migration processes ensuring data integrity, consistency, and performance.</li> <li>Work with Apache Spark, Scala, Airflow, Hadoop, and Python to manage and optimize data workflows.</li> <li>Collaborate with cross-functional teams to understand data requirements and migration goals.</li> <li>Develop and maintain detailed migration documentation, including data mappings, transformation rules, and processes.</li> <li>Troubleshoot and resolve any issues that arise during the migration process.</li> <li>Implement best practices for data management, storage, and retrieval.</li> <li>Ensure the security and compliance of data throughout the migration process.</li> <li>Use GitHub for version control and collaboration on code repositories.</li> </ul><p></p><h3>Qualifications</h3><ul> <li>Extensive experience with Apache Spark, Scala, Airflow, and Hadoop.</li> <li>Proficiency in Python for data engineering tasks.</li> <li>Strong experience with version control systems, particularly GitHub.</li> <li>Proven track record of leading successful data migration projects.</li> <li>Excellent problem-solving skills and the ability to troubleshoot complex issues.</li> <li>Strong communication and collaboration skills.</li> <li>Ability to work independently and lead a team in a fast-paced environment.</li> </ul><p></p><h3>What we offer</h3><ul> <li>20 days + public holidays.</li> <li>Ability to work remotely or join our tech hub in Warsaw.</li> <li>Help and support from our caring HR team.</li> <li>The exchange of experience and work with talented colleagues.</li> </ul><p></p>