Senior Software Engineer - CoreAI

Join our team within CoreAI, where we are building the AI data-plane that powers all LLM inferencing workloads across Microsoft and Azure customers—from cutting-edge startups to Fortune 500 enterprises. Our converged AI fabric delivers inference capabilities for all LLMs in Microsoft catalog, including OpenAI, Mistral, Cohere, Llama, and more. 

As a Senior Software Engineer - CoreAI, you will help build one of the largest and fastest-growing services in Azure, foundational to Microsoft’s AI strategy. Our mission is to serve models at scale—reliably, efficiently, and with ultra-low latency—enabling a rich set of AI-powered product experiences. This is a rapidly evolving space with immense opportunities to learn, innovate, and drive industry-wide impact. 

At Microsoft, our mission is to empower every person and every organization on the planet to achieve more. We embrace a growth mindset, foster innovation, and work together to realize our shared goals. Every day, we build on our values of respect, integrity, and accountability to create a culture where everyone can thrive. 

Responsibilities

• Drive innovation and deliver impact on a critical Azure service that underpins Microsoft’s AI vision. 
• Collaborate with a world-class team of engineers, architects, and leaders committed to technical excellence and career growth. 
• Innovate on product shape and offers to define how developers leverage cutting edge AI technologies and build world class applications. 
• Ship new product features and improvements at a high velocity where ideas to production is a matter of week(s).
• Design, implement and deliver AI services to support product offerings for large-scale LLM serving. 
• Innovate on technical solutions, and patterns that will improve the availability, reliability, efficiency, observability, and performance of products. 
• Engage with customers to gather feedback and resolve complex issues. 
• Embody our culture and values 

Qualifications

Required/Minimum Qualifications 

• Bachelor's Degree in Computer Science or related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java
  • OR equivalent experience. 
• 4+ years of design and problem-solving experience, with a deep understanding of system performance, scalability, and engineering best practices

Other Requirements: 

Ability to meet Microsoft, customer and/or government security screening requirements are required for this role. These requirements include, but are not limited to the following specialized security screenings: 

• Microsoft Cloud Background Check: This position will be required to pass the Microsoft Cloud background check upon hire/transfer and every two years thereafter.

Preferred/Additional Qualifications

• Understanding of distributed systems specifically in request serving at scale, including high-performance storage, distributed databases, and networking across global-scale infrastructures
• Experience shippin