Lead AI/Machine Learning Engineer

Overview

The AI/Machine Learning Engineer is focused on supporting data scientists by building structures to house, expose, and execute models. Often this is in support of enhancing human decision pipelines, so that humans can focus on the more creative and difficult tasks and models can drive automation on the "easy 80%". This is often a complex problem given the variability of systems at play in the credit union over the years. The data science team supports a mixture of batch and live models. The Lead AI/Machine Learning Engineer  is involved in the entire life cycle of the model from inception to monitoring, and if needed retirement. They work closely with Data Scientists to deploy models into various systems and Data Engineers to create the feature groups needed to support ongoing and future modeling needs, and do a lot of data engineering themselves. Their core focus and ownership within that pipeline is the tooling used to deliver models to production and server out to the greater enterprise in general. This includes building APIs and other types of interfaces. The end state should be that Data Scientists can self-serve as much as possible through the tooling and pipelines created by the Lead AI/Machine Learning Engineer.

 

Additionally, the Lead AI/Machine Learning Engineer will be responsible for the standards and practices upon which AI/MLOps is performed at the credit union. This will be accomplished in partnership with the Lead Data Scientist.

We do not model for the sake of modeling or to pad resumes. Everything is focused on working with a business owner to get the best (balancing complexity and accuracy) model into production and executing, even if that is a decision tree and not a neural network. We do not expect you to have all of the skills or exposure to all the tools listed below but be willing to work towards mastering them, and even help evolve their implementation (we do not know everything either). We put a heavy emphasis on being able to deploy models into production which includes writing tests, setting up monitoring, and performing code reviews.

 

Our culture can be summed up as:

• We all love building machine learning/AI systems
• We all have a life outside of work
• We challenge each other
• We prefer common tools and frameworks over niche tools and frameworks
• We prefer open source over proprietary, but do value the buy versus build judgement call when it is something we need but do not want as a core competency
• We value simplicity over complexity
• Always leave the code base better than you found it

What we value most in a team member is someone who can look at a vaguely defined problem and work iteratively in a collaborative fashion to find a clean solution.

 

In general, the cadence of the team follows agile rituals, however, we do not expect models to be completed within a sprint. Being a data scientific focused team, we understand that we do not know the answers, and that modeling can take a lot of twists and turns. So, to aid in planning and communication we utilize agile rituals but understand that modeling deliverables can be wildly different both in size and time to complete.

Responsibilities

• Lead. Lead the architecture, design, and implementation of end-to-end ML/AI systems. This will involve direct supervision of ML/AI Engineers including 1:1s, performance evaluations, and direct involvement in onboarding and offboarding of.
• Communicate. Collaborate with data scientists, data engineers, software engineers, and product managers to deliver impactful ML-driven features. The Lead AI/Machine Learning Engineer works as an internal consultant, consulting with business functions across different financial functions. Communicating, and being able to learn the language of that business unit, is critical to success of any Lead AI/Machine Learning Engineer. So financial experience is preferred, but not a must, as we have awesome product owners that are willing to help you along the way. Often the Lead AI/Machine Learning Engineer will be communicating directly with senior management.
• Mentor. Mentor and guide a team of ML/AI engineers and facilitate career development.
• Deploy Models. Deploy models in production environments, ensuring scalability, reliability, and maintainability. This will involve working with CI/CD pipelines, architecting the pipelines, and iterating on them to ensure efficient delivery.
• Monitor Models. Monitor model performance and continuously improve accuracy, latency, and efficiency. Given the stakes of a model making decisions without human intervention, a heavy emphasis is put on monitoring the models. This ranges everywhere from data metrics such as accuracy and recall to run-time and c