Data Engineer Job

Who are we

 

We are currently looking for a skilled Data Engineer to join our team and help us harness the power of cloud computing with AWS and Azure to transform data into actionable insights. 

 

What you'll be doing 

 

• Design, construct, install, test, and maintain highly scalable data management systems
• Ensure systems meet business requirements and industry practices
• Build high-performance algorithms, prototypes, predictive models, and proof of concepts 
• Research opportunities for data acquisition and new uses for existing data
• Develop data set processes for data modeling, mining, and production
• Integrate new data management technologies and software engineering tools into existing structures
• Create custom software components and analytics applications 
• Employ a variety of languages and tools to marry systems together
• Recommend ways to improve data reliability, efficiency, and quality
• Collaborate with data architects, modelers, and IT team members on project goals

What you'll bring along

 

• Bachelor’s Degree in Computer Science, Engineering, or related field
• At least 3-5 years of experience as a Data Engineer
• Strong knowledge and experience with AWS services like S3, Redshift, EMR, Glue, and AWS Lambda
• Proficiency in Azure services including Azure SQL Database, Azure Data Lake, HDInsight, and Azure Data Factory
• Experience with big data tools: Hadoop, Spark, Kafka, etc. 
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. 
• Experience with stream-processing systems: Storm, Spark-Streaming, etc. 
• Proficient understanding of distributed computing principles
• Proficiency with one or more relevant languages (Python, Scala, Java, etc.)
• Experience with relational SQL and NoSQL databases, including Postgres and Cassandra
• Strong analytic skills related to working with unstructured datasets
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores
• Experience with data visualization tools
• Strong project management and organizational skills