Data Engineer (AWS)

Addepto is a leading consulting and technology company specializing in AI and Big Data, helping clients deliver innovative data projects. We partner with top-tier global enterprises and pioneering startups, including Rolls Royce, Continental, Porsche, ABB, and WGU. Our exclusive focus on AI and Big Data has earned us recognition by Forbes as one of the top 10 AI companies.

As a Data Engineer, you will have the exciting opportunity to work with a team of technology experts on challenging projects across various industries, leveraging cutting-edge technologies. Here are some of the projects we are seeking talented individuals to join:

• Design and development of a universal data platform for global aerospace companies. This Azure and Databricks powered initiative combines diverse enterprise and public data sources. The data platform is at the early stages of the development, covering design of architecture and processes as well as giving freedom for technology selection.

• Development and maintenance of a large platform for processing automotive data. A significant amount of data is processed in both streaming and batch modes. The technology stack includes Spark, Cloudera, Airflow, Iceberg, Python, and AWS.

• Design of the data transformation and following data ops pipelines for global car manufacturer. This project aims to build a data processing system for both real-time streaming and batch data. We’ll handle data for business uses like process monitoring, analysis, and reporting, while also exploring LLMs for chatbots and data analysis. Key tasks include data cleaning, normalization, and optimizing the data model for performance and accuracy.

🚀 Your main responsibilities:

• Build and scale reliable data pipelines for both real-time and batch processing, using technologies like Airflow and AWS Step Functions.

• Develop automated data workflows that power analytics, reporting, and AI/ML models using services such as S3, Glue, EMR, Lambda, Redshift, and SageMaker.

• Work with diverse data sources, ensuring consistent quality, integrity, and readiness for analysis across platforms.

• Ingest, transform, and manage data from structured and semi-structured sources (including relational databases, cloud APIs, XML, JSON, and Hadoop/Spark).

• Partner closely with data science and business teams to align technical solutions with real-world use cases.

• Contribute to MLOps and CI/CD pipelines, helping streamline the deployment and monitoring of data products and models.

• Continuously evaluate and improve existing processes for speed, efficiency, and scalability in a cloud environment.

• Turn business problems into data-driven solutions, ensuring high performance, security, and scalability.

Requirements

🎯 What you'll need to succeed in this role:

• At least 3 years of professional programming experience implementing, developing, or maintaining Big Data systems, ideally in a business or enterprise setting.

• Strong proficiency in Python and SQL for data engineering tasks.

• Solid understanding of cloud data ecosystems, especially AWS.

• Familiarity with automation tools such as Airflow, AWS Step Functions, or similar workflow orchestrators.

• Experience working in Unix environments and using Git for version control.

• Excellent communication skills and consulting experience with direct interaction with clients.

• Ability to work independently and take ownership of project deliverables.

• Fluent in English (at least C1 level).

• Master’s in Computer Science, Big Data, Data Science, Mathematics, Physics, or a related field.

➕ Nice to have:

• Experience with cloud data warehouses like Redshift or Sno