Tag：gradient
-
[note] pytorch quick start: collection of basic parts
Pytorch quick start Tensors Tensors runs through pytorch Similar to multidimensional arrays, one feature is hardware acceleration Initialization of tensors There are many ways Direct value data = [[1,2],[3,4]] x_data = torch.tensor(data) From numpy array np_arr = np.array(data) x_np = torch.from_numpy(np_array) From another tensor x_ones = torch.ones_like(x_data) Assign 01 or random value shape = (2,3,) […]
-
AlexNet
catalogue Features of alexnet Using relu activation function to accelerate convergence How to understand the nonlinearity of relu function Paper: ImageNet Classification with Deep Convolutional Neural Networks Github：https://github.com/pytorch/vision/blob/master/torchvision/models/alexnet.py AlexNetThe model has the University of Toronto,Geoff HintonLaboratory design, won2012yearImageNet ILSVRCThe champion of the competition, and the error rate is far lower than the second, which makes […]
-
Data cache of “deep parsing” Ai training
The emergence of deep learning or AI has changed our previous programming method to solve problems, which is no longer an intuitive expression in code. For a simple example, how can we recognize that a number (picture) is the number 9? A very intuitive way is to have a small circle above and a vertical […]
-
Neural network and back propagation algorithm
catalog: 1. What is neural network 2. Forward propagation 3. Back propagation of error (BP algorithm) 4. Manually implement a basic fully connected neural network FCNN 5. Neural network practice – handwritten numeral recognition 6. Vectorization programming 1、 What is neural network Artificial neural network is referred to as “neural network”. In short, it is […]
-
Machine learning algorithm series (VII) – logarithmic probability regression algorithm (I) (logistic regression algorithm)
Background knowledge required for reading this article: linear regression, maximum likelihood estimation and yidui programming knowledge 1、 Introduction in the previous sections, we studied standard linear regression, and then introduced three regularization methods – ridge regression, Lasso regression and elastic network regression. These linear models solve the problem of regression. At the beginning, it […]
-
Machine learning algorithm series (VII) – logistic regression algorithm (I)
Background knowledge required for reading this article: linear regression, maximum likelihood estimation and yidui programming knowledge 1、 Introduction in the previous sections, we studied standard linear regression, and then introduced three regularization methods – ridge regression, Lasso regression and elastic network regression. These linear models solve the problem of regression. At the beginning, it […]
-
Four in-depth learning papers recommended for reading in January 2022
Time series prediction from bootstrap meta learning to deep learning, the relationship between extrapolation and generalization, and ridge rider to explore diversified optimization ‘Bootstrapped Meta-Learning’ Flennerhag et al. (2021) | 📝 https://arxiv.org/pdf/2109.04… The meta learning algorithm aims to automatically find inductive bias, which allows rapid adaptation in many tasks. Classic examples include MAML or RL […]
-
Improvement process of gradient descent method: from SGD to Adam algorithm
1. SGD gradient descent method 1.1 gradient descent Gradient GIt refers to the partial derivative of a function somewhere, pointing to the rising direction of the function. Therefore, the gradient descent method refers to updating the parameters with the negative number -g of the gradient, so as to make the next calculation result approach the […]
-
Machine learning algorithm series (VIII) – logarithmic probability regression algorithm (II) (logistic regression algorithm)
Background knowledge required for reading this article: logarithmic probability regression algorithm (I), conjugate gradient method, and a little programming knowledge 1、 Introduction The last article is the logarithmic probability regression algorithm (I), which introduces two methods to optimize the cost function of logarithmic probability regression – gradient descent method and Newton’s method. However, when using […]
-
Back propagation algorithm
1、 Overview The learning fitting ability of multi-layer network is much stronger than that of single-layer network. Therefore, if you want to train multi-layer networks, the previous simple perceptron learning method is obviously insufficient, and you need an algorithm with stronger fitting ability. Back propagation algorithm (BP) is one of the classical methods. It can […]
-
[mindspire: machine learning with little MI] learning of neural network (Part 2)
In the introduction given by Xiao Mi last week, we talked about how to use the back propagation algorithm to calculate the derivative of the cost function. Today, Xiao Mi will continue to show you the implementation process of neural network. There is no more nonsense. Learn it with Xiao Mi quickly~ 4. Implementation note: […]
-
[mindspire: let’s learn machine with little Mi!] Logistic regression (II)
Little MI, learn and be positive! The functional representation of logistic regression and the judgment boundary. Remember, don’t you think the last section is not strong?! No, little MI is updating again~ Cost functionNow we are going to discuss how to fit the parameters of logistic regression model. Specifically, we need to define the optimization […]