Senior Data Engineer — (all genders)

<h2 id="your-mission">Your mission</h2> <p>At Circunomics, we have made it our mission to provide innovative solutions for complete battery lifecycle management. Through battery analytics and a B2B marketplace, we are bringing industries together to encourage energy efficiency, reduce waste, and create a positive impact on the environment. Join our passionate team as we strive towards creating a true circular battery economy. </p> <ul> <li><strong>End-to-End Development:</strong> Design, develop, and deploy scalable, maintainable, efficient, and reliable applications using Python 3, FastAPI or Flask, with optional use of Angular. Ensure seamless integration of back-end components.</li> <li><strong>Big Data Management:</strong> Handle (near) real-time big data environments, managing and processing datasets starting from terabytes with precision and efficiency.</li> <li><strong>ETL Processes:</strong> Design, implement, and manage ETL (Extract, Transform, Load) processes to ensure efficient data handling through its entire lifecycle.</li> <li><strong>Performance Optimization:</strong> Implement caching services like Redis to improve application performance.</li> <li><strong>Data Pipelines:</strong> Develop and manage data pipelines using Airflow.</li> <li><strong>Mentorship:</strong> Provide guidance and mentorship to junior developers, fostering a culture of continuous learning and improvement within the team.</li> <li><strong>Architecture &amp; Design:</strong> Collaborate with architects and other developers to design robust, high-performance, secure application architectures.</li> <li><strong>Collaboration:</strong> Work closely with product managers, designers, and other stakeholders to understand requirements, provide technical insights, and ensure the successful delivery of features.</li> <li><strong>Code Quality &amp; Best Practices:</strong> Write clean, efficient, and well-documented code while setting and maintaining high standards for code quality through code reviews, testing, and automated tooling.</li> <li><strong>Containerization:</strong> Containerize applications using Docker and manage container orchestration with Kubernetes.</li> <li><strong>Infrastructure Management:</strong> Create and manage infrastructure using Helm Charts and Terraform.</li> <li><strong>Cloud Deployment:</strong> Deploy and manage applications on AWS cloud platforms, including S3, ECR, EKS, RDS, and EFS.</li> <li><strong>Data Visualization:</strong> Create data visualizations using Plotly.</li> <li><strong>CI/CD Implementation:</strong> Implement CI/CD pipelines using GitHub.</li> <li><strong>Troubleshooting &amp; Optimization:</strong> Troubleshoot and resolve application issues while optimizing performance.</li> </ul> <h2 id="your-profile">Your profile</h2> <ul> <li><strong>Experience:</strong> You have a minimum of 5 to 7 years of professional experience as a Data Engineer, particularly with Python 3, FastAPI, or Flask, and optional familiarity with Angular. You have a strong track record of designing and deploying scalable applications across backend components and, optionally, front-end components.</li> <li><strong>Big Data Management:</strong> You possess deep knowledge in managing (near) real-time big data environments, dealing with datasets starting from terabytes. You have implemented processes to efficiently handle, analyze, and process large volumes of data within strict deadlines.</li> <li><strong>ETL Processes:</strong> You are proficient in implementing ETL (Extract, Transform, Load) processes, ensuring effective and efficient data handling throughout its lifecycle.</li> <li><strong>Deep Understanding:</strong> You have in-depth knowledge of software development methodologies, RESTful APIs, microservices architecture, and CI/CD pipelines.</li> <li><strong>Problem-solving:</strong> You have strong analytical and problem-solving skills and can tackle complex technical challenges.</li> <li><strong>Communication:</strong> You have excellent communication and interpersonal skills and can convey complex technical concepts to non-technical stakeholders.</li> <li><strong>Adaptability:</strong> You thrive in a fast-paced, dynamic environment and can manage multiple priorities and projects simultaneously.</li> <li><strong>Language Skills:</strong> You have a good command of the English language.</li> <li><strong>Nice to Have:</strong> <ul> <li>Experience with Dash Framework for data visualization. </li> <li>Familiarity with other cloud platforms and technologies.</li> </ul> </li> </ul> <h2 id="why-us">Why us?</h2> <ul> <li>Flexible hours.</li> <li>Work-from-home options.</li> <li>Low hierarchies and open feedback culture.</li> <li>Open &amp; international environment.</li> <li>Modern offices are in the city center of Frankfurt.</li> <li>English is the company language and is used in international working environments.</li> <li>Attractive development perspectives with creative freedom for growth.</li> <li>Shape th