Lead Data Engineers – Azure/ Databricks/ Snowflake

Description

Lead Data Engineers – Azure/ Databricks/ Snowflake

Department:                          Sales and Delivery Team - Empower

Industry:                                 Information Technology & Services, Computer Software, Management Consulting

Location:                                 US Remote (US citizens, Candidates based in US with Valid Visa will be preferred)

Experience Range:                6 - 14 years

Basic Qualification:               Bachelor of Engineering or Equivalent

Travel Requirements:           Not required

Website:                                 www.exusia.com

 

 

Exusia, a cutting-edge digital transformation consultancy, and is looking for top Talent in Azure, Databricks Snowflake Data Warehouse Application Developers to join our global delivery team's Industry Analytics practice in India.

What’s the Role?

Full-time job to work with Exusia's clients to design and develop the system components for the extract and transform or conversion of data from various source systems to the target applications; Create data structures or data models that support data across all stages of the extract, transform and load process. Should have good hands on experience in implementing the parallel and sequence jobs using Snowflake and or or Azure or Databricks or Spark or Scala. Analyzing, developing, monitoring and administration of the ETL process using any tool. Experience working with on-shore/off-shore delivery model is a big plus.

The Lead Data Engineer will implement monitoring solutions, optimize ETL jobs for performance, and provide comprehensive support from data ingestion to final output. With in-depth knowledge of the Databricks platform and strong analytical skills, this position will significantly enhance the department's ability to deliver high quality, data-driven insights and solutions.

Criteria for the Role!

Have a minimum of 6+ years’ experience working as a Azure Databricks Developer in to a data analytics domain 

 

Azure Databricks Specific

•   4+ years of experience in Azure services and in Databricks (ADLS, ADF, DevOps, etc.)

•   10+ years of experience in designing and implementing ETL/ELT pipelines using ADF, integrating with other Azure services.

•   7+ years of Python development experience. Should have capability of developing own libraries

•   4+ years of experience in Snowflake or SQL experience (No-SQL experience is a plus)

•   Experience designing, building, and maintaining data processing systems

•   Experience with or knowledge of Agile Software Development methodologies

•   Education Qualification – Bachelor of Science (preferably in Computer and Information Sciences or Business Information Technology) or an Engineering degree in the above areas

Responsibilities:

·    Design, develop, and maintain data pipelines and ETL processes using Azure Data Factory, Azure Databricks

·    Build and optimize data storage solutions using Azure Data Lake, Azure SQL Database, and Azure Cosmos DB or Vector DB

 ·    Collaborate with data scientists, analysts, and business stakeholders to understand data requirements and deliver solutions.

·    Implement data quality checks, data governance, and security best practices across data platforms.

·    Monitor, troubleshoot, and optimize data workflows for performance and scalability.

·    Automate data integration and transformation processes using Azure DevOps and CI/CD pipelines.

·    Stay up-to-date with emerging Azure technologies and data engineering trends.

·    Deep understanding of data management, data governance, cloud and AI technologies, and their application in a business context.

·    Excellent problem solving and troubleshooting skills

•   Excellent oral and written communication skills

·    Strong analytical and problem-solving skills

·    Data experience working with multiple, large datasets / big data

·    Demonstrated experience in effectively working with multiple projects and activities in a timely, organized manner

·    Demonstrated experience in organizing and prioritizing work, meeting deadlines and working independently as well as collaboratively with others

·    Experience with Excel and PowerPoint.

Requirements

Nice – to – have Skills:

·       2+ years of experience with Azure stack example Azure Kubernet