In 2021, can we still start deep learning



During the Chinese new year, I had nothing to do with my own blog. I happened to see an article I had written before

This article was originally published in mid-2018, which would be exactly the sameDeep learning and neural network without brainWhen it’s time. How hot is it? When the graphics card was sold out (not because of mining), graduate tutors collectively recommended students to change the direction of deep learning. They could not finish their graduation thesis without the word “deep learning”. They also knew Wu Enda by asking any student on the street.

At this level of fire, my association resolutely stepped into the army of deep learning, began to pursue stars (Wu Enda, Li Hongyi), started classes (cs231n, cs229), began to learn framework (pytorch, tensorflow), and began to write water papers (escape ~).

In 2021, can we still start deep learning

But in the twinkling of an eye, 2021 has passed. What’s the matter now? In 2018, it will be known that hot search is already a “fight between the gods of algorithmic hillock”, 2019 is “twilight of the gods”, 2020 is “vanishing”, and I’m curious about what 2021 is. However, it is obvious that everyone has been cautious about artificial intelligence posts, especially CV posts.

In fact, the reason for this phenomenon is that CV is now in an awkward position where the threshold is low, there are more people looking for jobs and fewer jobs. On the one hand, there are a large number of students pouring in, on the other hand, there are sluggish economic environment and increasingly full jobs. Naturally, the employment difficulty of related posts reaches 18 levels.

However, this does not mean that the wave of deep learning has passed, but we are not so blind to deep learning as before.

The most flashy stage has passed, and the stage of boasting has passed. Now in this era, deep learning has become an infrastructure, gradually infiltrating into various industries and fields where we usually work. For example, face recognition has been basically mature (now the station basically realizes the verification of ID card and face through face recognition), speech recognition and OCR recognition technology are also applied.

This is the wave of intelligence and automation. What is difficult is not the CV volume, but how to not miss this era.

An article to solve the machine learning, deep learning entry doubts

The following text was written in 2018 and revised on February 15, 2021.

If you look at the news, artificial intelligence has been recognized by the state

In 2021, can we still start deep learning

It can be seen that our country attaches great importance to artificial intelligence. We should learn as soon as possible, and let’s make a contribution.

What this article is going to say

During the postgraduate period, I had an occasional chat with several senior students and several younger students. Most of the topics we talked about are not related to machine learning deep learning neural network, which has been burning in recent years. The purpose of this article is to look at this general direction and trend from the perspective of students (whether they are graduate students or undergraduate students about to enter graduate school), and also for those who have just started or are ready to start machine learning, deep learning, computer vision, image processing Voice recognition and processing and other related directions of children’s shoes, or research a children’s shoes to learn related directions, confused people clear some basic concepts, and solve these “nouns” represent what, how to choose, how to enter the problem, after all, who was also confused, more inheritance, less detours.

Major involved

There are more and more. As long as there is something to do with algorithms, almost all majors can engage in machine learning and deep learning neural networks. Take a few common majors for example: machine learning, deep learning, image processing, computer vision, big data processing, information retrieval, intelligent system, data mining, parallel computing, graphics and image processing Big data processing and visualization analysis

Two books on the desk, the classic watermelon book and the deep learning Bible.

In 2021, can we still start deep learning

Related applications involved

There are too many applications in machine learning and deep learning, such as image segmentation, language recognition, super-resolution, image tracking, image generation and so on. Specific to each small field, there are related applications. Here, I will briefly introduce two use scenarios.

  • Machine learning: for example, Taobao recommendation, one or two days of painstaking efforts to find, suddenly want to buy a game. After you open the mobile Taobao app, click search, enter the name of a game console (e.g. switch), and view the information of some game console stores. At this time, Taobao is already recording your information. No matter which online store you open, Taobao will collect your preferences. Taobao will record every link you open, and the content of each link will be recorded for you. When you log in to Taobao app next time, Taobao will analyze your hobbies and characteristics according to the information collected on your side and machine learning algorithm, and then push you some items that are more suitable for you and you may like.
  • Deep learning: another example is the popular face changing project – deepfake:

The feature information of one picture is extracted by variational self encoder, and then restored according to the feature information of another picture
In 2021, can we still start deep learning

The top three sides of each of these are the faces of trump, and the Nicola face on the far side, which is generated by encoding the features from the encoder. Through this, you can even confuse the video with the real. This is an application of deep learning. Those who are interested in this project can see its related articles.


First of all, the text may not be as much as you think. This article does not want to summarize and sort out all the knowledge of machine learning and deep learning, which is meaningless. Because, first, machine learning involves a lot of formula algorithms. It is not friendly for beginners to write so many formulas. It was originally a very simple thing to get started, and there is no need to complicate it. Second, the positioning of this article is neither machine learning nor deep learning textbooks. Of course, it is not a popular science article. This article is to talk about this topic from the perspective of college students, Share some of my experience with you.

Start to get down to business, no matter what you did before, what you learned, what your major is, what your field is, don’t worry. There are some basic programming (C / C + +), there are some basic knowledge of Mathematics (high numbers, linear algebra, probability and Statistics) can be, in addition, the main language of machine learning is python. No matter you are familiar with or not, you must have heard of Python. Yes, this language is very popular, powerful, powerful and omnipotent. Machine learning is closely related to python. Many deep learning libraries are programmed in Python language, so it is necessary to learn this language.

machine learning

First of all, it explains the relationship between machine learning and deep learning: deep learning is a subset of machine learning, and deep learning is a small part of machine learning. If you want to understand deep learning, you must first understand machine learning.

In 2021, can we still start deep learning

From the above figure, we can see the relationship among artificial intelligence, machine learning and deep learning. The scope of artificial intelligence is very wide, including not only machine learning, but also deep learning. At first, it’s easy to confuse. Although deep learning is a part of machine learning in theory, the main positioning targets of machine learning and deep learning can be distinguished. Machine learning mainly processes text data (data in tables), while deep learning mainly processes images and voice.

As for the history of machine learning, I don’t want to say much about it here. Just grab a bunch of them online and check them by yourself. Also take a look at the machine learning watermelon book (the picture above) above the introduction to the history of machine learning.

What is machine learning, in essence, is to use some mathematical algorithms to solve practical problems. And we’ve learned some of these mathematical algorithms. Students who have done mathematical modeling are more or less exposed to the knowledge of machine learning. When you use matlab to fit a function linearly, you are already using machine learning.

The general process of machine learning can be divided into three parts

  • Get the data you need (get data, organize data, enhance data)
  • Data analysis (data preprocessing)
  • Use the corresponding algorithm to process the data (machine learning algorithm)
  • And then the process of getting the results( Output results)

To be frank, if we think of the algorithm as a simple function (this function is the algorithm you designed)y = x + 2xIt’s input, it’s data,yIt’s the output, the result you want to get,x + 2This process is the process of your algorithm.

Let’s make it simple. Let’s make it a little more complicated5 = w*x + 4。

Isn’t this function an equation? Yes, it’s an equation. If we get the value of X, we can find w accordingly. But now we don’t think about it in terms of equations. Let’s think about it in terms of machine learning.

Let’s say that we can’t see the result at a glance. The above function is a machine learning algorithm designed by us, which makes the algorithm get a result by inputting the data X given by us5)Therefore, the above machine learning algorithm (function) is designed by ourselves. We want to get output 5 by inputting X.

So we designed:5 = w*x + 4

Then what to do? Machine learning is the process of machine learning. In machine learning, we want our designed algorithm to gradually learn how to get the final result. This process is obtained through training.

Training is definitely different from actual combat. Generally, in the process of machine learning, our data set is divided into two parts: training set and test set, which are equivalent to practice and actual combat. Of course, the training process is aimed atSupervised learningWhat is supervised learning? To put it simply, there are labels and marks. For example, let’s give you a picture with a cat on it, and the corresponding mark of this picture is 1. If this picture is a dog, then the corresponding mark of this picture is 2. Let’s let the machine learn to judge a series of pictures with cats and dogs on them, and give them the correct picture and corresponding mark during the training process. It can be understood as giving a few year old child a picture, giving him a picture of a cat, he said 1, and giving him a picture of a dog, he said 2. This kind of learning process is supervised. In the process of learning, you know whether your own judgment is correct, or someone supervises you and tells you what your judgment is wrong.

In 2021, can we still start deep learning

The right side of the picture above is the result of the algorithm, and the one with the highest score is the most likely result judged by the algorithm (the above judgment is obviously wrong. It was originally a cat’s picture, but the cat’s score is negative, while the dog’s score is 437. Obviously, the algorithm at this time mistakenly identifies the cat as a dog, and needs to continue “training”).

In unsupervised learning, there is no need to mark. It allows you to judge whether it is correct according to your own feelings. It is like giving you a pile of triangle, circle or rectangle cards to put them into three categories. You can divide these cards into three categories by distinguishing their shapes.

Back to our previous question:5 = w*x + 4

First of all, we don’t know what w is. Here we call w weight. Let’s assume that w is 1 (this is the initialization of weight, and there are many initialization methods), and then we require input x to be 2 (the input here is fixed, and 2 can only be 2. 2 here is equivalent to data obtained from life. For example, if you put a coin into the water vending machine, a bottle of mineral water will come out, and 1 here is the input, If you put in 0.5 or 2 coins, you will not get mineral water).

So, X is 2 and W is 1. This is our first attempt. We run the algorithm, and soon we find that,1*2 + 4 = 6. 6 is not equal to 5. Of course, the result is wrong.

Is wrong wrong wrong? In order to achieve the correct result, we need to design a standard, here we design a loss, hereinafter referred to as l, we let L = y – 5, which is the difference between our algorithm result and the correct result (here the correct result can also be called ground truth). The loss is the loss, and the function is the loss function. The loss function shows how far the result of our algorithm is from the actual result. When l is 0, it shows that our algorithm can get the desired result perfectly.

But obviously the L here is6-5=1If the loss is not zero, we calculate the derivative of the weight W for the loss L through the chain rule

dL/dw = dL/dy * dy/dw

Obviously we can get itDL / DW = 1 * 2 (x = 2)

So we get the derivative of loss to input. Here we define a learning rate: R (learning rate) and let it be 0.1. Then we get the gradient descent formula of X

w = w - r * dL/dw

Bring in the specific number:
w = 1 - 0.1 * 2 = 0.8

At this time, the weight W is 0.8.
Back to the original formula:w*x + 4 = 0.8 * 2 + 4 = 5.6

Although 5.6 is still not equal to 5, compared with the previous result of 6, it is a little close, as can be seen from L,L = y - 5 = 5.6 - 5 = 0.6After a “learning process”, the loss becomes smaller.

It’s easy to see that if the learning rate is changed to 0.2, then it will not lose more quickly? Of course, the learning rate is a super parameter (that is, a parameter set according to experience), which is set by us. It’s not good if it’s too high or too low, just appropriate.

Using a picture from cs231n:

In 2021, can we still start deep learning

Epoch is the number of training, different learning rate results in different, appropriate.

In this way, we only trained once to reduce the loss from 1 to 0.6. When we trained many times, the loss will continue to decline, but we need to pay attention to that, the loss will not “just” drop to 0, which is rare. In our future experiments, the loss will not change when it drops to a certain extent, indicating that the training is over.

So much for machine learning.

Deep learning

Deep learning is a part of machine learning, which can be summarized as neural network with deeper network layers.

There are many explanations of neural network on the Internet, so we will not repeat them. You can see the relevant information below.

Relevant information

There are many materials related to machine learning and deep learning. If you want to list them comprehensively, you can’t finish 100 pages. Here I only recommend some materials with high cost performance to beginners. In addition, there is no separation between machine learning and deep learning in theory, but since it is better to have a sequential process of learning, my suggestion is to first have a comprehensive understanding and practice of machine learning, and then carry out deep learning, so that the foundation will be very solid.


  • Recommended programmer’s mathematics series, a total of three (the first slightly chicken ribs, the latter two dry goods more), respectively, the programmer’s mathematics 1, linear algebra (programmer’s mathematics 2) and probability and Statistics (programmer’s Mathematics 3), these three can effectively supplement the linear algebra and probability and statistics knowledge we need.

In 2021, can we still start deep learning

  • “Zhou Zhihua – machine learning – Tsinghua University Press” can be used as a desk reference book, similar to teaching materials. It is boring to read chapter by chapter. It is recommended that you read it when you need it. There are many formulas and basic knowledge in the book…
  • It can be seen from the name of “machine learning actual combat”. This book mainly uses Python program to write machine learning related algorithms, with less explanation of formulas and more actual combat code…
  • Deep learning – Renming post and Telecommunications Publishing House is also a reference book for the desk. The content of the book is very detailed. The previous chapters introduced that machine learning is only the foundation for deep learning. After that, they began to explain a series of knowledge and formulas for deep learning. Most of the basic algorithms for deep learning are mentioned above. Pseudo code and formula coexist…
  • Professor Li Hang’s statistical learning method, which is combined with machine learning, can cover about 99% of the basic knowledge of machine learning. However, this book only explains some principles and formulas, and it is better to practice in combination with actual combat.

Video materials

  • Wu Enda series: according to the Internet, Wu Enda’s courses are generally divided into two parts: one is machine learning | coursera on coursera, and the other is Stanford University Course: s229: machine learning. These two courses explain the mathematical knowledge related to machine learning, and the former is simpler than the latter. This course involves more formulas but less engineering practice, It may seem boring to compare prices, but it’s very important to lay the foundation. If you are not very familiar with English or the video loading is slow (sometimes the video needs to climb over the wall), you can consider watching domestic transportation: Wu Enda’s first lesson of artificial intelligence, Netease Stanford machine learning Wu Enda open class. In addition, Wu Enda recently plans to publish a new book combining engineering practice and mathematical explanation, which is completely free of charge. You only need to subscribe to get it. Those who are interested can read it here.
  • Li Hongyi’s deep machine learning course at NTU:…, a well-known Chinese machine learning course. The lectures are very attentive, and the examples are vivid and interesting. The lectures are humorous, and the knowledge and interest coexist. It is also recommended.
  • Cs231n: deep learning courses at Stanford University: can be said that this is the best and most dry deep learning course in the whole network. It explains all the basic knowledge needed by neural network from the most basic classification algorithm. It also includes several classic neural network architectures (vggnet and RESNET) and several well-known deep learning applications in deep learning, and compares several deep learning frameworks. The most important thing is that, Homework after class is worth doing, it is difficult and can learn a lot of key points. All in all, this course is highly recommended.
  • In the series of Youda Learning City, the courses of Youda learning city are famous for their excellent engineering examples. The final projects of each course are mostly interesting and applicable, such as predicting house prices, classifying dog breeds, generating movie scripts, etc. There are a large number of courses, but each section is very short. There are questions to answer in each section, and there are many additional knowledge for you to learn. Although some knowledge explanations may not be very deep, they are very suitable for beginners. Just a little… These courses are not free. To put it simply, the deep learning course is divided into two semesters: 3299 yuan in the first semester and 3999 yuan in the second semester.

On the cost of learning

This is a commonplace question. There are different opinions about gods in the world. There are a thousand readers and a thousand Hamlets. I also encountered such a problem at the beginning. I have chosen a period of time to summarize it here.

Here are the answers for 2018:

If your direction is machine learning (when neural network is not involved, or the number of layers of neural network is not very high), the requirement of computer is not very high, ordinary notebook computer can. It doesn’t matter with or without a graphics card. But if your main direction is deep learning, and you need to process image or video information, especially when you design a neural network with many layers (deep layer is deep learning), the role of the graphics card will be reflected. The deep learning code of the graphics card is the choice suggestion for the graphics card (the front is the model, and the back is the size of the video memory): entry GTX 1060 6G The most cost-effective GTX 10708g, need to study GTX 1080ti 11g, of course, you can choose Titan graphics card (3w +) or multi graphics card, in short, wealth limits imagination. If you already have NVIDIA’s graphics card on your computer, it’s OK, but it depends on whether the computing power is enough (generally, if the capacity is greater than 4.0), and whether it can be used. Here, check whether the computing capacity of your graphics card is enough…
PS: because of various factors such as mining and so on, the price of graphics card is rising sharply. Should we buy it or have to buy it? We should transform wealth into productivity.
A CPU and GPU are attached to test the operation speed of deep learning…

Views in 2021:

I haven’t used the latest series 30, so I won’t comment on it. I’ve only used 2080ti for 20 series graphics card (half precision reasoning is very good), but what kind of graphics card to buy actually depends on the memory and computing characteristics (whether it supports single precision and half precision, etc.). If you want to buy it on demand, you can refer to an article I wrote before, which is still of reference significance:

Please tease me

  • If you and I are like-minded here, Lao pan is willing to communicate with you;
  • If you like Lao Pan’s content, you are welcome to pay attention and support.
  • If you like my article, I’d like to comment on it~

I want to know how Lao pan learned to step on the pits and want to communicate with me. Official account.Oldpan blog」。
Lao pan will also sort out some of his private collections, hoping to help you. ClickMysterious portalobtain.