Senior AI Scientist

What You Will Work On 

• Design and train large‑scale transformer and hybrid foundation models 

• Own model architecture choices across text, multimodal, and emerging paradigms 

• Diagnose and resolve training instabilities at scale 

• Navigate scaling tradeoffs across data and compute 

• Define the technical direction for next‑generation models 

Core Responsibilities 

Deep Learning & Transformer Foundations 

• Apply strong fundamentals in deep learning and representation learning 

• Design and modify transformer architectures, including: 

Attention variants 

• RoPE, ALiBi 

• Grouped Query Attention (GQA) 

• Mixture‑of‑Experts (MoE) 

• Build models from first principles, not just adapt pre‑existing codebases 

Optimisation Dynamics & Training Stability 

• Own optimiser and scheduler choices, including: 

• AdamW 

• Lion 

• Adafactor 

• Learning‑rate and warmup schedulers 

• Understand and debug: 

• Optimiser instability 

• Gradient pathologies 

Scaling Laws & Compute Tradeoffs 

• Apply and validate scaling laws 

• Navigate Chinchilla‑style compute vs data tradeoffs 

• Make informed decisions about model size, dataset size, and training duration 

Loss Functions & Alignment 

• Design and experiment with loss functions including: 

• Next‑token prediction 

• Contrastive objectives 

• RLHF, DPO, GRPO 

• Understand how loss design impacts convergence, generalization, and alignment 

Distributed Foundation Model Training 

• Design and execute large‑scale training using: