Cloud Data Engineer

Overview

AmTrust is seeking a highly motivated and accomplished Cloud Data Engineer with expertise in Azure Databricks, Azure Synapse Analytics, SQL, and cloud-based data platforms. The ideal candidate will have experience designing, developing, and optimizing scalable data lakehouse and data warehousing solutions that support enterprise reporting, analytics, and business intelligence. Strong technical skills in Python, PySpark, SQL, T-SQL, and Scala are required, along with experience building high-volume ETL/ELT pipelines, developing highly scalable and reusable Databricks notebooks, and leveraging Azure DevOps, Git, CI/CD, and cloud automation best practices to deliver secure and efficient data solutions. 

The successful candidate will also support AI and machine learning initiatives, including data preparation, feature engineering, Generative AI integrations, and MLOps processes.  

Responsibilities

• Design, develop, and maintain scalable cloud-based data engineering solutions, including ETL/ELT pipelines, data integration processes, reports, and analytics workflows using technologies such as Azure Databricks, Azure Synapse Analytics, Azure Data Factory, and SQL. 
• Demonstrate expert knowledge of modern cloud architecture, data lakehouse design, and data warehousing best practices, evaluating technical design alternatives and their business and operational impacts. 
• Develop, optimize, and troubleshoot complex SQL, T-SQL, Spark SQL, and PySpark workloads, with a strong focus on performance tuning, scalability, reliability, and cost efficiency. 
• Design, code, test, tune, deploy, and document new and existing data solutions, including Databricks notebooks developed in Python, PySpark, Spark SQL, and Scala, as well as cloud-native applications, APIs, and data pipelines utilizing Git, CI/CD, and Azure DevOps best practices. 
• Manage the complete development lifecycle for complex, high-impact data initiatives, including requirements gathering, solution design, development, testing, deployment, production support, and continuous improvement. 
• Support and enable AI, machine learning, and Generative AI initiatives by developing curated datasets, feature engineering pipelines, and scalable data platforms that power advanced analytics and intelligent business solutions. 
• Stay current with emerging trends and technologies in cloud computing, data engineering, artificial intelligence, Databricks, Azure services, automation, and analytics, recommending innovative solutions that drive business value and operational excellence. 
• Collaborate effectively with business stakeholders, architects, developers, and leadership teams, leveraging exceptional verbal and written communication skills to translate business requirements into scalable technical solutions and communicate complex concepts to both technical and non-technical audiences. 
• Perform other functionally related duties and responsibilities as assigned. 

Qualifications

• 7+ years of experience in cloud data engineering, ETL/ELT development, data warehousing, and enterprise analytics solutions. 
• Strong hands-on experience with Azure Databricks, Azure Synapse Analytics, Azure Data Factory, and modern cloud-based data lakehouse architectures. 
• Proven experience implementing the Medallion Architecture (Bronze, Silver, and Gold layers) to support scalable, governed, and high-quality data pipelines for enterprise analytics and reporting. 
• Advanced proficiency in Python, PySpark, Spark SQL, SQL, T-SQL, and Scala, with the ability to develop, optimize, and support highly scalable Databricks notebooks and distributed data processing workloads. 
• Proven experience designing, developing, and tuning complex queries and large-scale data processing solutions across high-volume datasets. 
• Experience implementing CI/CD pipelines, Git-based source control, Azure DevOps, and automated deployment frameworks. 
• Strong understanding of data warehousing concepts, including dimensional modeling, Star and Snowflake schemas, data governance, metadata management, and data quality best practices. 
• Experience supporting AI, Machine Learning, and Generative AI initiatives, including data preparation, feature engineering, model integration, vectorized data architectures, and MLOps processes. 
• Bachelor's degree in Computer Science, Information Systems, Engineering, Mathematics, Statistics, or a related technical discipline, or equivalent work experience. 
• Strong analytical and problem-solving skills with a demonstrated ability to design innovative solutions for complex business and technical challenges. 
• Experience working with large datasets, data cleansing, data transformation, reporting, statistical analysis, and advanced analytics. 
• Exceptional verbal and writte