Senior Machine Learning Engineer II

POS-29621

We're seeking a Machine Learning Engineer to join our Developer Experience team and own the operational deployment and performance optimization of our AI coding infrastructure. You'll be the expert who ensures our code generation models run reliably and efficiently at scale, powering the systems that help developers write software.

You'll work across the full model lifecycle. This includes fine-tuning open source models for code generation tasks and implementing RLHF pipelines to improve code quality and align with developer workflows, then taking those customized models and deploying them at scale. You'll evaluate and test bleeding-edge code models as they're released, debug distributed inference frameworks like vLLM, SGLang, and Ray, resolve GPU memory allocation issues, manage CUDA dependencies and kernel compatibility, and navigate the ever-shifting landscape of ML library ecosystems.

Your primary focus will be maximizing the throughput and capacity we can extract from our GPU infrastructure, turning experimental code models and fine-tuned variants into production-ready systems that generate code at scale for thousands of developers.

We are looking for people who have:

• A Boston-area location (or willingness to be local) and an interest in working in the office a couple of days each week.

• A track record of deploying and optimizing LLMs in production environments that serve users at scale

• Versatility to work across the stack in multiple languages (TypeScript, Java) as needed to integrate models into production systems

• Rigor to improve model performance through experimentation with inference configurations, fine-tuning, and RLHF

• Experience evaluating different models and inference frameworks to identify the best fit for specific latency, throughput, and quality requirements

• Ability to communicate complex technical tradeoffs to product and engineering teams

• Ownership mindset working cross-functionally to ship model improvements that directly impact developer productivity

• Curiosity to experiment with new model releases and appetite to debug the messy edge cases that come with production ML systems

Minimum Qualifications:

• Bachelor's degree in Computer Science or related field
• 5+ years of experience in machine learning or ML infrastructure roles
• Strong proficiency with LLM inference frameworks (vLLM, SGLang, Ray, or similar)
• Experience with Python and deep learning frameworks (PyTorch, TensorFlow)
• Hands-on experience with GPU optimization, CUDA, and distributed computing
• Experience with model evaluation, fine-tuning, or RLHF for LLMs
• Familiarity with MLOps best practices around model deployment, monitoring, and quality assurance

Preferred Qualifications:

• Master degree in a specialized discipline like Data Science or Machine Learning
• Experience specifically with code generation models or AI coding tools
• Background in developer tooling (static analysis, IDEs, development tools, etc…)
• Contributions to open source ML infrastructure projects

If you're passionate about leveraging machine learning to transform how developers build software, come join us at HubSpot!

Pay & Benefits

The cash compensation below includes base salary, on-target commission for employees in eligible roles, and annual bonus targets under HubSpot’s bonus plan for eligible roles. In addition to cash compensation, some roles are eligible to participate in HubSpot’s equity plan to receive restricted stock units (RSUs). Some roles may also be eligible for overtime pay. Individual compensation packages are tailored to your skills, experience, qualifications, and other job-related reasons.

This resource will help guide how we recommend thinking about the range you see. Learn more about HubSpot’s compensation philosophy.

Benefits are also an important piece of your total compensation package. Explore the benefits and perks HubSpot offers to help employees grow better.

At HubSpot, fair compensation practices aren’t just about checking off the box for legal compliance. It’s about living out our value of transparency with our employees, candidates, and community.

Annual Cash Compensation Range:$191,000—$305,600 USD

We know the