Sr. Data Engineer (Python/Spark)

Our client is a leading provider of financial and economic data to the investment banking industry. They are seeking an experienced Senior Data Engineer to join and growing team.

- This role is only available for W2 or individual contracts. Please no C2C.

- Remote 100%

Responsibilities:

• Analyze system requirements and design responsive algorithms and solutions.
• Build efficient ETL pipeline using Spark.
• Use big data and cloud technologies to produce production quality code.
• Engage in performance tuning and scalability engineering.
• Work with team, peers and management to identify objectives and set priorities.
• Perform related SDLC engineering activities like sprint planning and estimation.
• Work effectively in small agile teams.
• Provide creative solutions to problems.
• Identify opportunities for improvement and execute.

Requirements:

• 6+ years of proven professional experience working in the IT industry.
• AWS EMR experience. 
• Degree in Computer Science or related domains.
• Candidate must have 3-5 years of demonstrated experience in building big data solutions on AWS cloud primarily with PySpark, preferably in Data Analytics space.
• Minimum 3 - 5 years experience in application development using Python.
• Minimum 3 - 5 years of experience working with Spark
• Must have hands on experience developing data engineering solutions in Python using: S3, EMR, Glue, Athena, Kafka and notebooks.
• Must have hands on development experience in building distributed Big Data solutions including ingestion, caching, processing, consumption, logging & monitoring.
• Position requires strong technical communication skills.

Bonus:

• Big Data and Open Source technical experience.
• Agile (Scrum) methodology.
• Experience developing SaaS application backends and APIs using a variety of tools.
• Experience turning abstract business requirements into concrete technical plans.
• Proficiency with algorithms (including time and space complexity analysis), data structures, and software architecture. 
• Must be a quick learner to evaluate and embrace new technologies in the Big data space.

Thanks for applying!