Why AI models rarely make it to Production

栏目: IT技术 · 发布时间: 4年前

内容简介:Joint post withAfter making artificial intelligence investments, many teams have found the ROI to be elusive. Often, great experiments fail to become great products. That pain is summed up nicely in this tweet:After all the sweat equity invested in develop

Why AI models rarely make it to Production

How an MLOps mindset can revitalize your AI projects

Joint post with Tony Paikeday from NVIDIA.

Photo by Daniel Öberg on Unsplash

Problem: ML models are hard to ship

After making artificial intelligence investments, many teams have found the ROI to be elusive. Often, great experiments fail to become great products. That pain is summed up nicely in this tweet:

After all the sweat equity invested in developing models with high predictive accuracy, it’s painful to see businesses fall into “model debt” with un-deployed or under-deployed models. More often than not, what’s missing is an “ MLOps ” strategy that unifies machine learning development and IT best practices. Let’s look at why.

Why is getting ML models into production so difficult?

  1. Teams don’t implement the same DevOps rigor for data science projects as they do for more traditional dev projects. Like other production products, production AI pipelines should have monitoring, versioning, scalability, & repeatability. Organizations should budget for IT teams to implement this same level of resilience and repeatability for AI projects.
  2. AI models are not like conventional software. Model performance can degrade rapidly after the code has shipped. Continuous monitoring and retraining are required, and the related tools are still largely immature — due largely to the fact that best practices are still constantly evolving. Organizations should at least plan for continued data scientist involvement even after models are in production.
The unplanned steps to get to Production

How to plan for success

① People: Bring IT and data science teams together

A lot of AI work is opaque — not everybody can easily inspect or understand it. The people who build it are basically considered artisans. So, organizations often perceive that the responsibility for these projects lies solely with data science teams. This is short-sighted.

AI projects need to be integrated into the enterprise IT operation. AI developers often aren’t–and probably shouldn’t–be experts in how to build sturdy, scalable platforms.

And it’s a two-way street: both sides of the house need to plan for each other’s work. Do your data scientists build robust models? Do they have a scalable design for inference? Can your IT team support ad-hoc infrastructure requests? Do they provide self-service environments to developers? Maybe you need an “AI Engineer” role?

② Process: Implement a Design → Test → Production process for AI projects

Today, many companies haven’t implemented lifecycle management for their AI projects. Like with any product, having a defined release cycle standardizes the process. Having a little bit of process actually makes it easier for individual projects to get out the door since they don’t have to reinvent the wheel each time.

Expect for the process to evolve and grow — that’s a good sign that you’re using the system. But you have to start somewhere.

③ Technology: Plan for rapidly evolving technology

For traditional software development workflows, modern devops tools are constantly evolving. But for AI development, the tools are evolving even more rapidly as the application ecosystem rushes fill tooling gaps. Planning for evolving MLOps applications requires flexible infrastructure and a willingness to keep researching the latest schedulers, versioning tools, experiment trackers, ….

Do your data scientists and your infrastructure teams have the same understanding of the pipeline components?

Prioritize Production instead of Experimentation

The goal of AI is to use models to impact the business, not just to run experiments for experiments’ sake. So by prioritizing production over experimentation, we’re not saying don’t do necessary exploratory work — we’re saying that all work, including exploration, should have the end result in mind.

The fastest way to bridge the gap between experimentation and production is to bridge the gap between AI teams and IT teams. Acknowledge the lift required beyond model development. Commit to implementing the same DevOps rigor for data science projects as you do for traditional dev projects.


以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网

查看所有标签

猜你喜欢:

本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们

游戏之旅

游戏之旅

云风 / 电子工业出版社 / 2005-12-01 / 46.00

这是一本非常有特色的计算机编程学习书籍。其特色就在于它将作者十余年来对游戏编程的所思、所感、所悟与编程理论知识相结合,褪去了纯理论的教学理念,使读者在前人的学习过程中吸取学习经验和教训,将计算机基础知识和高级编程技术不知不觉地融入自己的头脑中。 本书忠实地记录了作者十余年来对游戏编程的所思、所感、所悟。全书按照作者本人学习和实践的过程,带着读者从基础的计算机知识到高级的编程技......一起来看看 《游戏之旅》 这本书的介绍吧!

MD5 加密
MD5 加密

MD5 加密工具

RGB CMYK 转换工具
RGB CMYK 转换工具

RGB CMYK 互转工具