# Sigmoid function

Time：2021-9-14

## preface

Sigmoid function (logistic function) is a very common activation function in neural networks. Let’s have a deep understanding of sigmoid function today.

## code implementation

Code run: colab

``````import matplotlib.pyplot as plt
import numpy as np
import math

x = np.linspace(-10, 10, 100)
z = 1 / (1 + np.exp(-x))

plt.title("Sigmoid")
plt.plot(x, z)
plt.xlabel("x")
plt.ylabel("Sigmoid(X)")

plt.savefig("sigmoid.png")
plt.show()
plt.close()``````

## Nature and problems

The value range of function value s (x) is (0, 1), which is often used in binary classification problems. The function is smooth and easy to derive. However, as an activation function, it has a large amount of calculation. When calculating the error gradient by back propagation, there is division in the derivation, which is prone to the disappearance of the gradient. When the input is close to positive infinity or negative infinity, the gradient approaches 0 and gradient dispersion occurs (with the increase of the number of network layers, when using the back propagation algorithm to calculate the gradient, the gradient disappears very obviously from the output layer to the first few layers, resulting in the derivative of the overall loss function to the weight of the first few layers is very small. Therefore, when using the gradient descent algorithm, the weight of the first few layers changes very slowly, and even useful features cannot be learned.) . because the sigmoid function value is greater than 0, the weight update can only be updated in one direction, which may affect the convergence speed.

## summary

Sigmoid function is a very common activation function in neural networks. It is widely used in logistic regression and in the fields of statistics and machine learning.

• Original from: rais

## Beautify your code VB (VBS) code formatting implementation code

However, vb.net does have many new functions that VB6 does not have. The automatic typesetting of code is one, which is the function we want to realize today – VB code formatting.Let’s look at the effect: Before formatting: Copy codeThe code is as follows: For i = 0 To WebBrowser1.Document.All.length – 1 If WebBrowser1.Document.All(i).tagName = […]