内容简介:Joint post withAfter making artificial intelligence investments, many teams have found the ROI to be elusive. Often, great experiments fail to become great products. That pain is summed up nicely in this tweet:After all the sweat equity invested in develop
Why AI models rarely make it to Production
How an MLOps mindset can revitalize your AI projects
Joint post with Tony Paikeday from NVIDIA.
Problem: ML models are hard to ship
After making artificial intelligence investments, many teams have found the ROI to be elusive. Often, great experiments fail to become great products. That pain is summed up nicely in this tweet:
After all the sweat equity invested in developing models with high predictive accuracy, it’s painful to see businesses fall into “model debt” with un-deployed or under-deployed models. More often than not, what’s missing is an “ MLOps ” strategy that unifies machine learning development and IT best practices. Let’s look at why.
Why is getting ML models into production so difficult?
- Teams don’t implement the same DevOps rigor for data science projects as they do for more traditional dev projects. Like other production products, production AI pipelines should have monitoring, versioning, scalability, & repeatability. Organizations should budget for IT teams to implement this same level of resilience and repeatability for AI projects.
- AI models are not like conventional software. Model performance can degrade rapidly after the code has shipped. Continuous monitoring and retraining are required, and the related tools are still largely immature — due largely to the fact that best practices are still constantly evolving. Organizations should at least plan for continued data scientist involvement even after models are in production.
How to plan for success
① People: Bring IT and data science teams together
A lot of AI work is opaque — not everybody can easily inspect or understand it. The people who build it are basically considered artisans. So, organizations often perceive that the responsibility for these projects lies solely with data science teams. This is short-sighted.
AI projects need to be integrated into the enterprise IT operation. AI developers often aren’t–and probably shouldn’t–be experts in how to build sturdy, scalable platforms.
And it’s a two-way street: both sides of the house need to plan for each other’s work. Do your data scientists build robust models? Do they have a scalable design for inference? Can your IT team support ad-hoc infrastructure requests? Do they provide self-service environments to developers? Maybe you need an “AI Engineer” role?
② Process: Implement a Design → Test → Production process for AI projects
Today, many companies haven’t implemented lifecycle management for their AI projects. Like with any product, having a defined release cycle standardizes the process. Having a little bit of process actually makes it easier for individual projects to get out the door since they don’t have to reinvent the wheel each time.
Expect for the process to evolve and grow — that’s a good sign that you’re using the system. But you have to start somewhere.
③ Technology: Plan for rapidly evolving technology
For traditional software development workflows, modern devops tools are constantly evolving. But for AI development, the tools are evolving even more rapidly as the application ecosystem rushes fill tooling gaps. Planning for evolving MLOps applications requires flexible infrastructure and a willingness to keep researching the latest schedulers, versioning tools, experiment trackers, ….
Prioritize Production instead of Experimentation
The goal of AI is to use models to impact the business, not just to run experiments for experiments’ sake. So by prioritizing production over experimentation, we’re not saying don’t do necessary exploratory work — we’re saying that all work, including exploration, should have the end result in mind.
The fastest way to bridge the gap between experimentation and production is to bridge the gap between AI teams and IT teams. Acknowledge the lift required beyond model development. Commit to implementing the same DevOps rigor for data science projects as you do for traditional dev projects.
以上就是本文的全部内容,希望本文的内容对大家的学习或者工作能带来一定的帮助,也希望大家多多支持 码农网
猜你喜欢:本站部分资源来源于网络,本站转载出于传递更多信息之目的,版权归原作者或者来源机构所有,如转载稿涉及版权问题,请联系我们。
MD5 加密
MD5 加密工具
RGB CMYK 转换工具
RGB CMYK 互转工具