Senior Engineer II, AI Inference Engine Systems

<div class="content-intro"><p>Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you’ll find your place here. &nbsp;We value winning together—while learning, having fun, and making a profound difference for the dreamers and builders in the world.&nbsp;</p></div><p>DigitalOcean is expanding its AI Infrastructure layer to support the next generation of AI-driven applications. We are seeking a <strong>Senior Engineer 2</strong> to join our AI Inference Engine Systems&nbsp;team. In this role, you will be a key technical leader responsible for designing, developing, and delivering high-scale, resilient data plane services that power our "Inference as a Service" offering. You will work at the intersection of distributed systems and specialized AI hardware to ensure our customers can deploy and scale their models with industry-leading performance and reliability. This is a hands-on role, requiring you to be able to develop high quality software while availing of all the productivity boosts granted by the latest AI coding agents.&nbsp;</p> <h2><strong>What You’ll Do:</strong></h2> <ul> <li><strong>Technical Leadership:</strong> Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models.</li> <li><strong>System Design:</strong> Architect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standards.</li> <li><strong>Performance Optimization:</strong> Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing.</li> <li><strong>Collaboration:</strong> Work cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs.</li> <li><strong>Mentorship:</strong> Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement.</li> <li><strong>Operational Excellence:</strong> Maintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform health.</li> </ul> <h2><strong>What You’ll Bring to DigitalOcean:</strong></h2> <ul> <li><strong>Distributed Systems Expertise:</strong> Strong experience with microservices, messagin