Why AI models rarely make it to Production

栏目: IT技术 · 发布时间: 4年前

内容简介:Joint post withAfter making artificial intelligence investments, many teams have found the ROI to be elusive. Often, great experiments fail to become great products. That pain is summed up nicely in this tweet:After all the sweat equity invested in develop

Why AI models rarely make it to Production

How an MLOps mindset can revitalize your AI projects

Joint post with Tony Paikeday from NVIDIA.

Photo by Daniel Öberg on Unsplash

Problem: ML models are hard to ship

After making artificial intelligence investments, many teams have found the ROI to be elusive. Often, great experiments fail to become great products. That pain is summed up nicely in this tweet:

After all the sweat equity invested in developing models with high predictive accuracy, it’s painful to see businesses fall into “model debt” with un-deployed or under-deployed models. More often than not, what’s missing is an “ MLOps ” strategy that unifies machine learning development and IT best practices. Let’s look at why.

Why is getting ML models into production so difficult?

  1. Teams don’t implement the same DevOps rigor for data science projects as they do for more traditional dev projects. Like other production products, production AI pipelines should have monitoring, versioning, scalability, & repeatability. Organizations should budget for IT teams to implement this same level of resilience and repeatability for AI projects.
  2. AI models are not like conventional software. Model performance can degrade rapidly after the code has shipped. Continuous monitoring and retraining are required, and the related tools are still largely immature — due largely to the fact that best practices are still constantly evolving. Organizations should at least plan for continued data scientist involvement even after models are in production.
The unplanned steps to get to Production

How to plan for success

① People: Bring IT and data science teams together

A lot of AI work is opaque — not everybody can easily inspect or understand it. The people who build it are basically considered artisans. So, organizations often perceive that the responsibility for these projects lies solely with data science teams. This is short-sighted.

AI projects need to be integrated into the enterprise IT operation. AI developers often aren’t–and probably shouldn’t–be experts in how to build sturdy, scalable platforms.

And it’s a two-way street: both sides of the house need to plan for each other’s work. Do your data scientists build robust models? Do they have a scalable design for inference? Can your IT team support ad-hoc infrastructure requests? Do they provide self-service environments to developers? Maybe you need an “AI Engineer” role?

② Process: Implement a Design → Test → Production process for AI projects

Today, many companies haven’t implemented lifecycle management for their AI projects. Like with any product, having a defined release cycle standardizes the process. Having a little bit of process actually makes it easier for individual projects to get out the door since they don’t have to reinvent the wheel each time.

Expect for the process to evolve and grow — that’s a good sign that you’re using the system. But you have to start somewhere.

③ Technology: Plan for rapidly evolving technology

For traditional software development workflows, modern devops tools are constantly evolving. But for AI development, the tools are evolving even more rapidly as the application ecosystem rushes fill tooling gaps. Planning for evolving MLOps applications requires flexible infrastructure and a willingness to keep researching the latest schedulers, versioning tools, experiment trackers, ….

Do your data scientists and your infrastructure teams have the same understanding of the pipeline components?

Prioritize Production instead of Experimentation

The goal of AI is to use models to impact the business, not just to run experiments for experiments’ sake. So by prioritizing production over experimentation, we’re not saying don’t do necessary exploratory work — we’re saying that all work, including exploration, should have the end result in mind.

The fastest way to bridge the gap between experimentation and production is to bridge the gap between AI teams and IT teams. Acknowledge the lift required beyond model development. Commit to implementing the same DevOps rigor for data science projects as you do for traditional dev projects.


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

iGen

iGen

Jean M. Twenge PhD / Atria Books / 2017-8-22 / USD 27.00

A highly readable and entertaining first look at how today’s members of iGen—the children, teens, and young adults born in the mid-1990s and later—are vastly different from their Millennial predecesso......一起来看看 《iGen》 这本书的介绍吧!

HTML 编码/解码
HTML 编码/解码

HTML 编码/解码

XML 在线格式化
XML 在线格式化

在线 XML 格式化压缩工具

Markdown 在线编辑器
Markdown 在线编辑器

Markdown 在线编辑器