Technical Specialist-Data Engg

Area(s) of responsibility

Job Description:

We are seeking a highly skilled and motivated Data Engineer to join our data team. As a Data Engineer, you will be primarily responsible for designing & developing data transformations and data models to ensure reliable and efficient data processing and analysis. You will work closely with cross-functional teams to support data-driven decision-making processes and contribute to the overall success of our insight’s teams.

Key Proficiency & Responsibilities:

• Expertise in DBT (Data Build Tool) for data transformation and modelling.
• Proficiency in Snowflake, including experience with Snowflake SQL and data warehousing concepts.
• Strong programming skills in Python for data manipulation, automation, and integration tasks.
• Proficiency in using StreamSets for building and managing scalable data pipelines.
• Strong understanding of data architecture, data modelling, and data warehousing best practices.
• Implement and optimize data ingestion processes to ensure efficient and accurate data flow from various sources.

Essential job tasks

• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and ensure data integrity and quality.
• Perform data analysis and profiling to identify and address data quality issues.
• Maintain comprehensive documentation of data workflows, models, and ETL processes.
• Stay up to date with the latest trends and technologies in data engineering, DBT, and Snowflake.
• Experience with other ETL tools and technologies is a plus (e.g., Apache Airflow, Talend, Informatica).
• Proficient in SQL and experience with programming languages such as Python or Java.
• Familiarity with cloud platforms and services (e.g., AWS) and experience on AWS Lambda is a must.
• Adhere to development best practices, including version control using Git and branching models.
• Participate in scrum methodology, including daily stand-ups, sprint planning, and retrospectives.
• Communicate effectively with team members and stakeholders to understand requirements and provide updates.
• Take ownership of assigned tasks and work independently to complete them.