Google recently released an open source “model search” platform, which can help researchers develop machine learning models efficiently and automatically.
Google said that model search does not focus on a specific area and can find a model architecture suitable for data sets and problems, while minimizing coding time and computing resources.
Based on tensorflow machine learning framework, it is composed of many algorithms
The success of an AI model usually depends on its performance under various workloads. However, it is very challenging to design a model that can be well summarized. In recent years, automl algorithm began to appear to help researchers find the right model without manual experiments. However, in general, these algorithms are computationally expensive and require thousands of models to train.
Model search is based on tensorflow machine learning framework of Google. It can run on a single machine or multiple machines. It is composed of multiple trainers, search algorithm, transfer learning algorithm and database storing evaluation model. It runs training and evaluation experiments of artificial intelligence model in an adaptive and asynchronous way, so that all trainers can share the knowledge gained from their experiments and conduct each experiment independently. At the beginning of each cycle, the search algorithm finds all the completed experiments and decides what to try next. Then it “mutates” one of the best architectures found so far, and returns the resulting model to the trainer.
Model search provides a flexible and domain agnostic framework for machine learning model discovery
In order to further improve the efficiency and accuracy of model search, transfer learning is introduced into the experiment. For example, it uses knowledge extraction and weight sharing, which leads some variable models from previous training models. This enables faster training and wider opportunities to discover more and better architectures.
After the model search runs, users can compare many models found during the search. In addition, they can create their own search space to customize the architectural elements in the model.
Google said that in an internal experiment, model search improved the production model with a minimum of iterations, especially in terms of keyword recognition and language recognition. It also managed to find an architecture suitable for major exploration of image classification in cifar-10 open source imaging dataset.
In a blog post, Google Research Engineer Hanna Mazzawi and research scientist Xavi gonzalvo wrote, “we hope that model search will provide researchers with a flexible and domain agnostic machine learning model discovery framework. By building prior knowledge in a specific domain, we believe that when providing search spaces composed of standard building blocks, the framework is powerful enough to build models with the most advanced performance for in-depth research.