Recently, Google officially announced that tensorflow 0.8 has fully realized parallel computing and the combination of original ecological support and kubernetes, which makes the open-source project tensorflow of Google machine learning achieve a qualitative leap in supporting clustering, parallelism and distributed training.
In Google’s official blog last week, Google announced the acceleration effect that different numbers of GPUs of tensorflow 0.8 can bring:
The figure shows that 100 GPUs can bring nearly 56 times acceleration effect, and train the image classifier to nearly 78% accuracy within 65 hours.
TensorSlow or TensorFlow?
Let’s review the history of tensorflow. As early as the end of last year, Google opened its deep learning system tensorflow. And according to GitHub, tensorflow has become one of the most popular open source projects last year. At the beginning of its open source, it attracted numerous trial users. However, the newly open source tensorflow has not attracted much praise. It is also jokingly called tensorflow. The reason is that the single machine environment cannot meet the needs of massive data.
In contrast, as early as 2011, Google began to study deep learning, and now it has been successfully applied to search, advertising, maps, translation, youtube and many other products. Through in-depth learning, Google has successfully reduced the error rate of speech recognition by 25%. And Google brain (tensorflow in Google’s internal version) has long realized the combination with Borg (kubernetes in Google’s internal version), which can easily run on tens of thousands of machines.
Last month, tensorflow took the first step of clustering (original link) and realized parallelization through kubernetes.
Now, Google officially announced that tensorflow 0.8 can support distributed training, and provide kubernetes test script as well as multi GPU test results. With the combination of tensorflow and kubernetes, it will have a profound impact on the practice of machine learning technology.
What is deep learning?
Let’s take a look at “deep learning”. The word “deep learning” has become a familiar concept for many people. This concept has become more well known after alphago defeated Li Shishi. What is deep learning?
The essence of deep learning is to mine the hidden rules in the complex data through the complex model structure or the combination of multi-layer nonlinear changes. In theory, any complex machine learning method can be used as the basis of deep learning, but most mature applications and research are based on neural network.
Influenced by the structure of human brain neurons, neural networks hope to simulate human perception. One of the most direct examples is image recognition, which is also one of the earliest applications of deep learning. After human eyes are stimulated by light, different neurons in the brain will be activated. Finally, we can tell whether the object in front of us is a dog or a cat.
In neural networks, neurons are used to simulate the function of human neurons. The figure above shows a neuron model: one (or more) outputs are generated by a series of transformations of multiple inputs. The figure below shows how to effectively organize multiple neurons into a neural network. The red input layer simulates the human organs (such as eyes) receiving stimulation, and the blue hidden layer (multiple hidden layers) simulates the processing of human brain. Finally, the green output layer gives a conclusion (such as cat or dog).
Neural network algorithm was proposed as early as the 1970s, but it has not been paid attention to for a long time, until recently, it has become a new favorite through deep learning. So what’s the difference between deep learning and traditional neural network?
Their biggest difference is in depth. The hidden layer of traditional neural network is generally less (generally 1-2 layers), and the hidden layer of deep learning is deeper. Microsoft Research Institute has tried thousands of layers of neural network.
In theory, the deeper the expression ability of neural network is, the more likely it is to find hidden rules from complex problems. However, the deeper the neural network, the higher the requirements for data and computation, which is why tensorflow needs to be combined with kubernetes to bring more profound impact.
(if you need to reprint, please contact us. Everyone has the responsibility to respect intellectual property rights.)