Google Believes Machine Learning Frameworks Need Five Key Things to Reach Mainstream Developers

Google Believes Machine Learning Frameworks Need Five Key Things to Reach Mainstream Developers

Google Research conducted a detailed survey within TensorFlow.js developers to determine the key elements that streamline adoption of machine learning frameworks.

Machine learning(ML) could be the most important element of the next generation of software applications and, yet, its usage its constrained to high skilled developers. Differently from previous technology trends, machine learning is not showing a clear transition path into mainstream developer adoption. Part of that friction is due to the fact that most machine learning frameworks still require a very high entry point in terms of computer science knowledge. Recently, Google Research published a paper that outlined five key principles to break that friction point and design machine learning frameworks that can be adopted by a broader range of developers.

Tinkers vs. Scientists: The Impostor Syndrome in Machine Learning

In a famous article published in 1978, Dr. Pauline R. Clance described a phenomenon in which accomplished individuals experience of self-perceived intellectual phoniness or the feeling of being a fraud. Dr. Clance referred to this phenomenon as the “impostor syndrome” which has become a common term in modern psychology. The impostor syndrome is present across all sorts of areas of a modern society but its definitely prevalent in scenarios that require a highly intellectual skills. I believe a version of that phenomenon is negatively influencing the adoption of machine learning technologies as many developers believe they don’t have the necessary mathematical and computer science skills to jump into the space.

The industry to software development technologies have been a constant quest to lover the entry point to increase adoption. From the early programming languages to recent movements such as mobile developments, software development technologies undergoes several simplification cycles to attract broader groups of developers. Since the early days of programming languages and graphical interfaces, most software development stacks have created layers upon layers of abstractions that hide the underlying computer science details. Part of the massive developer adoption experienced by mobile and web technologies relied on attracting developers many of which are “tinkers” and doers without a computer science background. That path towards simplification doesn’t seem very obvious in the case of machine learning.

Google’s research correctly uncovered that many developers fear to get into machine learning technologies because they lack the mathematical skills required. Despite plenty of evidence of self-taught hackers tinkering with machine learning models, most developer see machine learning specialists as people with deep knowledge of linear algebra and statistics; a text-book definition of the impostor syndrome. In my opinion, there are several factors that are not helping to mitigate this challenge:

1) Mathematical terminology in ML frameworks documentation: Today, the documentation of most machine learning frameworks reads like a math class rather than a developer framework.

2) Acceleration of artificial intelligence(AI) research: The rapid growth of AI research is a fascinating thing. However, very regularly machine learning frameworks are forced to adapt new research techniques that lack a proper developer experience.

3) Lack of ML lifecycle management tools: The process debugging or testing machine learning models feels like a never ending exercise of tuning hundreds of obscure parameters that are not understood by mainstream developers. The existing toolset is the machine learning space is still too limited for mainstream adoption.

Google’s Five Principles for Better ML Frameworks

Designing better machine learning frameworks to streamline developer adoption is one of the pivotal challenges of the next decade of the AI space. Based on their research, Google identified five key aspects that machine learning framework designers should consider to lower the entry points for developers:

1) Demystify Mathematical and Algorithmic Concepts

Despite ongoing efforts to lower the barriers to machine learning, and despite a preponderance of ML-newcomers aspiring to learn, surprisingly many developers perceived current resources to be intended for more advanced audiences. New machine learning frameworks should abstract these mathematical models into digestible, practical concepts. For instance, most developer have no idea what a gradient-descent is, but they can certainly understand tuning the learning rate of a model.

2) Support Learning by Doing

Today, most tutorials that highlight best practices in machine learning require substantial lines of code. Abstracting best practices into small code-blocks that can be used by beginner developers is essential to help developers who prefer a “learning by doing” approach to mater a new technology.

3) Support Re-use and Modification of Pre-made ML Models

Complementing the previous point, seems important to include canonical models that could help developers casually become aware of machine learning idiosyncrasies from within their existing programming workflow. In addition to lower the entry point for developers, it can also help to enforce the consistency across programs.

4) Synthesize ML Best Practices into Just-in-Time Hints

Decisions such as which parameter to fine-tune first or how many layers to use in a model can require years of machine learning development experience. Building better visualization and interpretability tools that use strategic pointers can help developers overcome this challenge.

5) Emphasize and Support the Experimental Nature of ML

The lifecycle of machine learning applications is different from previous technology trends. Specifically, machine learning applications require heavy experimentation and trial and error. Despite using the best model for a specific scenario, it is required to conduct dozens and dozens of experiments in order to ensure its optimal performance. Facilitating the experimentation process should be another key focus on future machine learning frameworks.

These are some of the recommendations Google Research believes are required to streamline the adoption of machine learning frameworks. Ultimately, new developer stacks need to balance incorporating new research methods with a simple developer experience that facilitates the adoption by a broader range of developers.