• ## Extending regression library by regularization

Time：2021-7-9

By Paul HiemstraCompile VKSource: towards Data Science You can also read this article on GitHub. This GitHub repository contains everything you need to run your own analysis:https://github.com/PaulHiemstra/lasso_tsfresh_article/blob/master/lasso_tsfresh_article.ipynb introduce For many data scientists, the most basic model is multiple linear regression. In many analyses, it is the first to be invoked and serves as a benchmark […]

• ## Reproducing kernel Hilbert space

Time：2021-6-4

Hilbert space Let’s first talk about what isHilbert space。 This concept sounds tall, but it’s actually a very simple concept.First of all, what is itlinear space linear space Linear space is the space that defines multiplication and addition. This is a space with a linear structure. With the concept of linear space, we can find […]

• ## Data dimension reduction: principal component analysis

Time：2021-4-19

preface What is called principal component analysis? Let’s first look at a graph of an ellipse. If you were asked to find a line so that all the points on the ellipse mapped on the line were the most scattered and the most information remained, how would you choose this line? In the figure below, […]

• ## L1L2 regularization

Time：2021-3-31

P-norm The definition of p-norm $\|\mathbf{x}\|_{p}:=\left(\sum_{i=1}^{n}\left|x_{i}\right|^{p}\right)^{1 / p}$ When p = 1, $\ | – boldsymbol {x}\|_ {1}:=\sum_ {i=1}^{n}\left|x_ {i}\right|$When p = 2, $\ | – boldsymbol {x}\|_ {2}:=\left(\sum_ {i=1}^{n}\left|x_ {i}\right|^{2}\right)^{\frac{1}{2}}$L1 norm is the sum of the absolute values of each element in the vector, L2 norm is the sum of the squares of each […]

• ## Regularization in deep learning (1)

Time：2021-3-17

This article starts with the official account number: RAIS, click direct attention. preface This series of articles are reading notes of deep learning, which can be read together with the original book for better effect. In this paper, we talk about regularization in deep learning. Regularization in deep learning Generally speaking, what deep learning needs […]

• ## Basic mathematics linear algebra

Time：2021-1-29

The content of linear algebra is very coherent, and the whole is [determinant — > matrix — > n-dimensional vector — > system of linear equations — > similar diagonal type — > quadratic type]. The determinant is a value. If the determinant is 0, the corresponding linear equations have multiple solutions, and the corresponding […]

• ## Algorithm Engineering II. Mathematical basis linear algebra

Time：2020-10-30

Linear algebra content is very coherent, the whole is [determinant > matrix > n-dimensional vector > linear equations system > similar diagonal type > quadratic form]. The determinant is a value. If the determinant is 0, the corresponding linear equations have multiple solutions, and the corresponding matrix is irreversible. If the determinant is 0, the […]

• ## Optimization in sparsenn

Time：2020-10-29

The AI LabCompile | VKSource | medium This paper studies the over fitting of sparsenn model, and explores a variety of regularization methods, such as Max norm / constant norm of embedded vector, dropout of sparse feature ID, freezing of parameters, embedding shrinkage, etc. However, as far as we know, in a single training, there […]

• ## Intuitive understanding of regularization in deep learning

Time：2020-10-21

By Kelvin LeeCompile | FlinSource: towards science Get an intuitive understanding of regularization In machine learning, regularization is a way to combat high variance – in other words, model learning reproduces the problem of data rather than the underlying semantics of the problem. Similar to human learning, the idea is to construct homework questions to […]

• ## L1 and L2: loss function and regularization

Time：2020-7-31

As a loss function L1 norm loss function 　　L1 norm loss functionAlso known as the minimum absolute error. Overall, it takes the target value of $y_ I$and estimate $f (x_ i)$ofabsolute difference Minimize the sum of. $$S=\sum_{i=1}^n|Y_i-f(x_i)|$$ L2 norm loss function 　　L2 norm loss functionAlso known as the least square error, in general, […]

• ## SVM support vector machine notes 1

Time：2020-7-14

Support vector machine Many lines in this diagram can separate the two points. But which one?Intuitively, we will definitely choose the middle one, because it splits the two parts of data most “open” and has the highest fault tolerance rate. The most marginal point here actually plays a very important role. As long as the […]

• ## Regularization, standardization and normalization in spark data processing

Time：2020-7-7

A. Regularization1. What is the purpose of regularization?The purpose of regularization is to prevent over fitting. The parameter estimation with the sum of regularization terms is in line with our previous target, that is, to fit the data with as few variables as possible. Regularization is in line with the principle of Occam razor. Among […]