Amazon Gets Into the AutoML Race with AutoGluon: Some AutoML Architectures You Should Know ...

Amazon, Microsoft, Salesforce, Waymo have produced some of the most innovative AutoML architectures in the market.

A few days ago, Amazon announced the release of AutoGloun , a new toolkit that simplifies the creation of deep learning models with just a few lines of code. The release marks Amazon’s entrance in the ultra-competitive Automated machine learning(AutoML) space which is becoming one of the hottest trends for enterprise machine learning platforms. With some many news around the AutoML ecosystem, sometimes it becomes hard to differentiate signal from noise. Today, I would like to explore some of the most innovative AutoML stacks in the market that don’t receive that much publicity.

AutoML is becoming one of the most popular topics in modern data science applications. Often, people see AutoML as a mechanism to use out-of-the-box machine learning models without the need of sophisticated data science knowledge. While theoretically, this argument makes sense the reality if a bit different. In the current stage of artificial intelligence(AI), most real world applications require some level of machine learning knowledge. The scenarios that you can solve with a vanilla API like the Watson Developer Cloud or Microsoft Cognitive Services are very basic and represent only a small percentage of the broader spectrum of machine learning scenarios. If that’s the case, then we should wonder what’s the real value of AutoML.

The Challenge

Model selection is one of the most difficult aspects of building a machine learning solutions. Somewhat ironically, for all the science and match baked into machine learning applications, model selection remains a highly subjective tasks that relies on the opinions of experts. For any given scenario, the number of machine learning models that can solve it is incredibly large so how can we really know if we are using the most optimal model for the job? Even worse, even if we have selected the right machine learning technique, how can we be sure we have the right neural network architecture in place? And once we settled on a specific architecture, how can we know the correct hyperparameter configurations? Those questions hunt data scientists throughout the entire lifecycle of a machine learning application. Furthermore, the more accuracy is needed for a machine learning problem the more time is spent in the model selection process.

Not surprisingly, the process of selecting and architecting a machine learning model is an extremely time-consuming exercise that never delivers an exact answer. Paradoxically, this is the type of problem in which machine learning excels so can we get creative and model the process of selecting a machine learning architecture as machine learning problem in and out itself?

AutoML to the Rescue

Model search is one of the use cases that seems like a perfect fit for AutoML. Given a dataset, a series of optimization metrics and some constraints in terms of time or resources, AutoML methods should be able to evaluate tens of thousands of neural network architectures and produce an optimal result. While effective data science teams might be able to evaluate a dozen models for a given problem, an AutoML method can quickly search through tens of thousands of architectures in a relatively manageable time.

Using machine learning to build better machine learning models seems like something out of an IronMan movie :wink: Is this really happening in the real world? Absolutely! Here are some of my favorite high-profile case studies for AutoML in mission critical applications.

AutoGluon

Let’s start with the newest member of the party: Amazon’s AutoGluon. Functionally, AutoGluon is an open-source library for developers building applications involving machine learning with image, text, or tabular data sets. AutoGluon enables easy-to-use and easy-to-extend AutoML with a focus on deep learning and real-world applications spanning image, text, or tabular data. The framework is intended for both machine learning beginners and advanced experts. The first version of AutoML includes some of the following capabilities:

· Quickly prototype deep learning solutions for your data with few lines of code.

· Leverage automatic hyperparameter tuning, model selection / architecture search, and data processing.

· Automatically utilize state-of-the-art deep learning techniques without expert knowledge.

· Easily improve existing bespoke models and data pipelines, or customize AutoGluon for your use-case.

Salesforce.com TransmogrifAI: The Brain Behind Einstein

Salesforce.com’s Einstein is one of the most widely adopted machine learning applications worldwide. Ultimately, Einstein solves a series of machine learning scenarios such as sales forecasting or lead prioritization which are omnipresent in sales and marketing applications. However, what makes Einstein unique is the fact that it machine learning models are able to operate across completely diverse Salesforce.com configuration in a self-service manner. Each customer might have completely different sales and marketing schemas and yet Einstein can still do the job.

The magic behind Salesforce’s Einstein is powered by an open source framework called TransmogrifAI . Conceptually, TransmogrifAI is an AutoML based framework for creating machine learning models against structure datasets(rows and columns). More specifically, TransmogrifAI leverages AutoML is five fundamental areas of a machine learning workflow:

· Feature Inference: Extracting features from a given datasets.

· Transmogrification: Transforming features into numeric values.

· Feature Validation: Reduce dimensions, identify potential bias, etc.

· Model Selection: Conducting search across thousands of potential models.

· Hyperparameter Optimization: Tuning the hyperparameter configuration.

Given the Salesforce.com footprint, TransmogrifAI might be considered one of the largest AutoML applications in the world.

Azure ML: Helping Developers Select the Right Machine Learning Model

Last year, Microsoft Research conducted an experiment of leveraging AutoML and probabilistic programming for automating model selection. The results were captured in a very popular research paper and represented a breakthrough from the performance. In a matter of months, the AutoML approached pioneered by the Microsoft Research team were implemented in Microsoft’s hallmark machine learning product: Azure ML.

The latest release of Azure ML leverage AutoML to streamline model selection. The platform includes an AutoML service that regularly recommends new machine learning pipelines to evaluate given a specific problem. The execution of the pipeline is done in the customer’s Azure ML instance while the AutoML service only sees the results and uses them to make better recommendations.

The AutoML implementation in the Azure ML stack is one of the most complete I’ve ever seen. The current version supports classification and regression ML model recommendation on numeric and text data, with support for automatic feature generation (including missing values imputations, encoding, normalizations and heuristics-based features), feature transformations and selection. Developers can use AutoML through the Python SDK or via Jupyter Notebooks.

Waymo: Automated Model Selection for Self-Driving Vehicles

A self-driving vehicle is something like a big group of machine learning models on four wheels :blush:. Machine learning enable all the intelligent features of self-driving vehicles like helping cars see their surroundings, make sense of the world, predict how others will behave, and decide their next best move. Alphabet’s subsidiary Waymo is at the forefront of self-driving vehicle technologies and, as a result, is constantly innovating in the machine learning.

Recently, the Waymo engineering team published a detailed blog post of how they are leveraging AutoML to automate model selection on different machine learning applications . Specifically, the Waymo team leverages an AutoML technique known as NAS cells which has proven to be very effective in image analysis algorithms.

At Waymo, AutoML is used to explore hundreds of different NAS cell combinations within a convolutional net architecture (CNN), training and evaluating models for Waymo’s LiDAR segmentation task. The experiment has been producing CNN architectures that are performing with 20–30% less latency and 8–10% lower error rate that hand-crafted models.

As you can see from these examples, AutoML is gaining a position as one of the important elements of highly scalable machine learning architectures. Amazon’s entrance in the space is certainly another push towards make AutoML one of the key components of machine learning architectures. AutoGluon is another example that the tools and frameworks for leveraging AutoML in model searching are getting better and are becoming available to mainstream developers. While there are other great use cases for AutoML, model selection remains the one driving the biggest benefits in the context of real world machine learning applications.