Senior Principal Software Engineer

Company Description

At Red Hat, we connect an innovative community of customers, partners, and contributors to deliver an open source stack of trusted, high-performing solutions. We offer cloud, Linux, middleware, virtualization, and AI technologies, together with award-winning global customer support, consulting, and implementation services. Red Hat is a rapidly-growing company supporting more than 90% of Fortune 500 companies.

Job Summary

Red Hat's Global Engineering Team is looking for a Senior Principal Software Engineer to join our newly formed AI Engineering organization. This role will be located within the RHEL AI Core Components team, and will be focused on inferencing platform strategy and architecture, as well as virtual large language model (vLLM) productization as part of our RHEL AI product. This will involve participating in the development of novel techniques to support vLLM and establishing Red Hat leadership in upstream vLLM communities. 

You will need to develop and maintain working relationships across multiple teams, planning and prioritizing work across a small team, along with direct contribution to development projects at a senior level of ability. This role will also require deep connections across Red Hat and IBM Research, collaborating with other engineering teams within Red Hat and peer IBM Research scientists to develop AI the open source way. 

This is a high-growth area of Red Hat’s business, so being effective in a fast-growing team and contributing to defining skill sets needed to build a cohesive team will be critical. This role requires a combination of technical expertise, strategic thinking, and strong technical leadership and influence skills. Successful candidates for this role must have strong knowledge of Red Hat’s customers as well as Red Hat’s product portfolio and emerging technologies. As this is a fast-moving area of opportunity for Red Hat, the ability to communicate productively and effectively with team members, stakeholders, and Red Hat leadership is critical.

This position reports directly to the Senior Manager of Software Engineering for RHEL AI Core Components. Successful applicants must reside in a state where Red Hat is registered to do business.

Primary Job Responsibilities (what you’ll do)

• Drive and coordinate contributions to vLLM as Red Hat’s strategic inferencing server. 

• Act as a bridge between platform research and product teams, supporting the delivery of platform components to IBM watsonx.ai, Red Hat OpenShift AI and others.

• Connect related work being done across different IBM research teams, helping to channel research into core components of Red Hat’s AI products.

• Design and develop scalable and efficient architectures for training and deploying large language models.

• Participate in and lead upstream data science and machine learning project communities, building Red Hat’s credibility and influence in these communities.

• Identify and resolve complex issues related to model training, inference, and system performance.

• Collaborate with other engineers, data scientists, and researchers to align on goals and product requirements.

• Implement optimizations for model training and inference to improve performance, efficiency, and resource utilization.

• Run experiments, tests, and large-scale distributed jobs in support of AI product features.

• Mentor junior engineers and provide guidance on complex technical issues and best practices.

Required Skills (what you’ll bring)

• Bachelor's degree in computer science or equivalent.

• Advanced programming skills in Python and SQL.

• Expertise in deep learning frameworks (e.g., TensorFlow, PyTorch) and architectures, particularly for natural language processing (NLP).

• Expertise with vLLM and other inference runtimes. 

• In-depth understanding of the design, training, and deployment of LLMs.

• Experience with the design and implementation of scalable and efficient system architectures for training and deploying large models.

• Knowledge of microservices and containerization technologies (e.g., Kubernetes) for deployment.

• Strong self-motivation and organizational skills.

• Demonstrated ability to context switch between multiple concurrent pro