Senior AI Data Scientist
Beth Israel Lahey HealthAbout the role
When you join the growing BILH team, you're not just taking a job, you’re making a difference in people’s lives.
The Senior AI Data Scientist role serves as a technical leader for complex machine learning and generative AI initiatives within the organization. This role is responsible for designing, fine-tuning, evaluating, and governing large and small language models (LLMs) and advanced machine learning models used in healthcare and operational settings. The Senior AI Data Scientist owns model quality, evaluation rigor, and responsible AI outcomes, while mentoring junior team members and guiding project-level technical direction. The position emphasizes model development, experimentation strategy, evaluation frameworks, and applied generative AI. The Senior AI Data Scientist partners closely with engineering, clinical, and data teams to ensure AI solutions are accurate, safe, explainable, and fit for production use.Job Description:
Essential Duties & Responsibilities including but not limited to:
1. Lead the design, development, and evaluation of advanced machine learning and generative AI models across healthcare use cases.
2. Serve as a subject matter expert on large and small language models (LLMs), including architecture tradeoffs, inference behavior, strengths, and limitations.
3. Design and oversee LLM fine-tuning strategies, including instruction tuning and parameter efficient fine-tuning approaches, ensuring alignment with domain needs and model governance standards.
4. Define and execute model evaluation frameworks for LLMs, including quantitative metrics, qualitative review, bias analysis, hallucination assessment, and safety testing.
5. Guide and review prompt design and optimization strategies, ensuring robustness, consistency, and interpretability of generative AI outputs.
6. Lead and improve retrieval of augmented generation (RAG) solutions, including data selection, grounding quality, relevance evaluation, and retrieval performance analysis.
7. Conduct advanced exploratory data analysis, feature engineering, and statistical modeling to support predictive and prescriptive analytics.
8. Translate complex clinical or business problems into clear modeling strategies and evaluation plans.
9. Mentor entry level and midlevel data scientists through model reviews, best practice guidance, and collaborative problem solving.
10. Partner with data engineering, software engineering, and clinical teams to ensure AI models are deployable, monitored, and compliant with governance requirements.
11. Define and promote Responsible AI best practices, including fairness, bias mitigation, transparency, documentation, and appropriate clinical use.
12. Communicate model behavior, tradeoffs, and outcomes to technical leaders, clinicians, and executive stakeholders.
Minimum Qualifications:
Apply for this role
Generate a tailored application kit with a matched cover letter, interview prep, and CV highlights — in under 60 seconds.
Apply Now →Generate Application KitFree account required — sign up in 30s