Cloud Engineer - Azure

Role : Cloud Engineer

location : Dallas TX ( Coppell office - Hybrid, NO REMOTE)

Blue Yonder is a leading supply chain software company that provides intelligent solutions for businesses worldwide. Our mission is to help our clients optimize their supply chain operations through cutting-edge technology and data-driven insights. We’re looking for an experienced Azure engineers to join a team tasked with automating and bringing efficiencies to product delivery on the Azure platform.

The Execution Machine Learning team works closely with sales, product, and engineering teams to design and implement the next-generation retail solutions. Data Science team members are tasked with turning both small, sparse, and massive data into actionable insights with measurable improvements to the customer bottom line. They use rigorous analysis and repeatable processes to implement black box and interpretable models. 

Our machine learning platform ingests data in real-time, processes information from millions of retail items to serve deep learning models, and produces billions of predictions daily 

Summary:

As a Cloud Engineer, you will be responsible for designing, deploying, configuring, and maintaining large-scale Apache Flink and Apache Beam clusters on Kubernetes within the Azure cloud platform. You will collaborate with data engineers, data scientists, and other stakeholders to ensure these clusters meet performance, scalability, and availability requirements in a Kubernetes and Azure environment.

Responsibilities:

Provision and manage Azure resources required for Apache Flink and Apache Beam clusters, including Azure Kubernetes Service (AKS) clusters, Storage, and Networking components.

Collaborate with architecture teams to design Flink and Beam clusters optimized for Kubernetes and Azure.

Implement orchestration tools to automate repetitive tasks and streamline workflows.

Define Kubernetes configurations, including pods, services, and ingress controllers.

Deploy, configure, and maintain Apache Flink and Apache Beam clusters on Kubernetes.

Optimize resource utilization and cost efficiency through capacity planning and performance tuning.

Implement automation scripts, Helm charts, and Kubernetes Operators for cluster deployments and updates.

Set up Azure Monitor and Log Analytics to monitor the performance and health of Flink and Beam clusters.

Create custom monitoring and alerting solutions using Azure Monitor.

Monitor cloud infrastructure for security vulnerabilities and take appropriate actions to mitigate risks.

Configure auto-scaling and resource optimization strategies to handle varying workloads efficiently.

Implement Azure Auto scale for dynamic resource allocation.

Provide training to team members on managing these clusters within the Azure environment.

Collaborate with data engineering, data science, and Azure cloud teams to align cluster architecture with business objectives.

Communicate effectively with stakeholders regarding the status, issues, and enhancements of the Azure-based Flink and Beam clusters.

Identify and resolve technical issues related to cloud infrastructure and services.

Document cloud architecture, processes, and procedures for knowledge sharing and future reference.

Qualifications:

Bachelor's or higher degree in Computer Science or a related field.

5+ years of experience building high-performance distributed systems, solving large-scale engineering challenges.

Proven experience in setting up and managing Apache Flink and Apache Beam clusters on Kubernetes at scale.

2 + years experience in running Data pipelines on Apache Flink at scale.

Strong understanding of Kubernetes concepts, including pod management, ingress controllers, and stateful sets.

Proficiency in Azure AKS and Azure Kubernetes integration.

Kubernetes certifications (e.g., Certified Kubernetes Administrator) are a plus.

Experience with DevOps practices and tools for Kubernetes.

Excellent problem-solving and communication skills.

Preferred Skills:

Certification in Apache Flink and Apache Beam or related technologies.

Familiarity with other Azure services and cloud-native technologies.

Knowledge of stream processing and big data technologies.

-------------------------------------------