• Extending regression library by regularization


    By Paul HiemstraCompile VKSource: towards Data Science You can also read this article on GitHub. This GitHub repository contains everything you need to run your own analysis:https://github.com/PaulHiemstra/lasso_tsfresh_article/blob/master/lasso_tsfresh_article.ipynb introduce For many data scientists, the most basic model is multiple linear regression. In many analyses, it is the first to be invoked and serves as a benchmark […]

  • Reproducing kernel Hilbert space


    Hilbert space Let’s first talk about what isHilbert space。 This concept sounds tall, but it’s actually a very simple concept.First of all, what is itlinear space linear space Linear space is the space that defines multiplication and addition. This is a space with a linear structure. With the concept of linear space, we can find […]

  • Data dimension reduction: principal component analysis


    preface What is called principal component analysis? Let’s first look at a graph of an ellipse. If you were asked to find a line so that all the points on the ellipse mapped on the line were the most scattered and the most information remained, how would you choose this line? In the figure below, […]

  • L1L2 regularization


    P-norm The definition of p-norm $\|\mathbf{x}\|_{p}:=\left(\sum_{i=1}^{n}\left|x_{i}\right|^{p}\right)^{1 / p}$ When p = 1, $\ | – boldsymbol {x}\|_ {1}:=\sum_ {i=1}^{n}\left|x_ {i}\right|$When p = 2, $\ | – boldsymbol {x}\|_ {2}:=\left(\sum_ {i=1}^{n}\left|x_ {i}\right|^{2}\right)^{\frac{1}{2}}$L1 norm is the sum of the absolute values of each element in the vector, L2 norm is the sum of the squares of each […]

  • Regularization in deep learning (1)


    This article starts with the official account number: RAIS, click direct attention. preface This series of articles are reading notes of deep learning, which can be read together with the original book for better effect. In this paper, we talk about regularization in deep learning. Regularization in deep learning Generally speaking, what deep learning needs […]

  • Basic mathematics linear algebra


    The content of linear algebra is very coherent, and the whole is [determinant — > matrix — > n-dimensional vector — > system of linear equations — > similar diagonal type — > quadratic type]. The determinant is a value. If the determinant is 0, the corresponding linear equations have multiple solutions, and the corresponding […]

  • Algorithm Engineering II. Mathematical basis linear algebra


    Linear algebra content is very coherent, the whole is [determinant > matrix > n-dimensional vector > linear equations system > similar diagonal type > quadratic form]. The determinant is a value. If the determinant is 0, the corresponding linear equations have multiple solutions, and the corresponding matrix is irreversible. If the determinant is 0, the […]

  • Optimization in sparsenn


    The AI LabCompile | VKSource | medium This paper studies the over fitting of sparsenn model, and explores a variety of regularization methods, such as Max norm / constant norm of embedded vector, dropout of sparse feature ID, freezing of parameters, embedding shrinkage, etc. However, as far as we know, in a single training, there […]

  • Intuitive understanding of regularization in deep learning


    By Kelvin LeeCompile | FlinSource: towards science Get an intuitive understanding of regularization In machine learning, regularization is a way to combat high variance – in other words, model learning reproduces the problem of data rather than the underlying semantics of the problem. Similar to human learning, the idea is to construct homework questions to […]

  • L1 and L2: loss function and regularization


    As a loss function L1 norm loss function   L1 norm loss functionAlso known as the minimum absolute error. Overall, it takes the target value of $y_ I $and estimate $f (x_ i) $ofabsolute difference Minimize the sum of. $$S=\sum_{i=1}^n|Y_i-f(x_i)|$$ L2 norm loss function   L2 norm loss functionAlso known as the least square error, in general, […]

  • SVM support vector machine notes 1


    Support vector machine Many lines in this diagram can separate the two points. But which one?Intuitively, we will definitely choose the middle one, because it splits the two parts of data most “open” and has the highest fault tolerance rate. The most marginal point here actually plays a very important role. As long as the […]

  • Regularization, standardization and normalization in spark data processing


    A. Regularization1. What is the purpose of regularization?The purpose of regularization is to prevent over fitting. The parameter estimation with the sum of regularization terms is in line with our previous target, that is, to fit the data with as few variables as possible. Regularization is in line with the principle of Occam razor. Among […]