INTERN - DevOps and Data Engineering / Scientist - Spark, Azure

Job Title

INTERN - DevOps and Data Engineering / Scientist - Spark, Azure

English version

Context

SSP division (Search, Shopping and Pricing) in Amadeus implement services for the creation of offers, fix prices and put in sale traveling solutions. SSP take the responsibility to developpe technologies for the search of prices and pilot new product on the flying industry. The architecture of SSP is based on C, C++ and Java. It allows the provisioning of a functional platform, scalable, adaptable and robust: which is needed for today’s industry.  

Within SSP, the DMD (Devops, Monitoring & Data platform) department contributes to the design of the system architecture and the creation of a high-performance application framework based on Unix/Linux. Our architecture now supports more than 100 different applications with a very large choice of options. The daily traffic processed by the SSP servers in production represents more than 1 billion transactions per day. The raw data produced by these transactions is saved and processed on a Azure cloud and KPIs are then made available in a distributed Elastic Search database. This architecture makes it possible to monitor the use of SSP products in real time and to improve their quality by delivering current and relevant content to our customers.  

The goal of the internship is to build a part on the pipeline that we will allow to compare 2 different solutions for the processing and make a benchmark on the price/performance. The first one being Spark on AKS and the second is Databricks provided by Azure. The intern will be responsible for deploying spark on AKS (k8s) to be able to run machine learning/ processing job. 

Objectives: 

• State of the art: Spark job on the cloud  

• Deploy Spark on AKS  

• Adapt the framework provided by the team to run spark job on AKS 

• Comparison and benchmark on the performance/cost of the 2 solutions. 

Common accountabilities:

- Performs routine and clearly defined tasks following specific instructions or under close supervision​

- Identifies the problem and all relevant issues in straightforward situations, assesses each using standard procedures and makes sound routine and learned decisions to take action.​

- Has limited working knowledge and skills developed through formal Training or work experience.

Specific accountabilities:

 
Having good communication skills.
• Being able to work under pressure.
• Having good judgement.
• Being trustworthy and reliable.
• Having high attention to details.
• Being able to act in a confidential manner.
• Having teamwork skills.
• Having interpersonal abilities.
• Being organised and methodical.
• Having commercial awareness.

Diversity & Inclusion

We are an Equal Opportunity Employer and seek to hire the best candidate regardless of age, beliefs, disability, ethnicity, gender or sexual orientation.