Kalman filter Kalman filter of Xiaobai’s AI


The article is from WeChat official account.
First of all, a photo of Kalman is given to show our respect
Kalman filter Kalman filter of Xiaobai's AI

Kalman filter

  • English Kalman filter
  • Here we introduce a simple filter with only one state
  • Kalman filter is often used in control system and robot system, but this paper mainly explains how to use it in AI big data analysis and prediction

Why use Kalman filter

Suppose we have 100 time points of data, and this data is the result of observation at 100 points.

There are two methods to obtain the data at each time point

  • The first one is observation, but the measurement results are not necessarily accurate, which may be limited by the accuracy of the measuring instruments?
  • The second is to use all the data before this time point to predict the data at this time point. Of course, the predicted value is also inaccurate.
  • Can we use these two methods to promote each other, so that the predicted values are more accurate, or that the observed values are closer to the essence? This is what the Kalman filter does.

There are bound to be people who don’t understand that the observed values are not always accurate. How can you rely on the predicted values? (in fact, this is what Ali’s interviewer refuted me. At that time, I was really deceived because this method was only used in feature construction, but the filter was used in control system specialty.)

Breaking the conceptual shackles of cognition: what has Kalman filtering done

What the Kalman filter does is: for example, we know the position of the aircraft at the last moment, and know the position of the aircraft measured by the radar at this moment. Use the first two data to estimate the position of the aircraft at this time. In short, it is to know the state of the last moment and the measurement data, and fuse the two data to find the current state.

You will askNow that you know the measurement data at the current moment, I think the current state is the measurement data, isn’t it good? In other words: “you must think that the position of the aircraft measured by the radar is not the current position of the aircraft? Why use Kalman filter to estimate the current position of aircraft.

answer: the position of the aircraft measured by the radar signal received at this moment is not necessarily the actual position of the aircraft. First of all, there is error in radar signal measurement. Second, if you think about it, I’m getting a radar signal now, which is a signal that was sent before and then returned. Does this process take time? During this period of time, the plane may fly at a speed of more than twice the speed of sound, or it may crash directly.That is to say, even if the measured data is received, it is still uncertain where the aircraft position is。 Therefore, I need to estimate the current aircraft position according to the position of the previous moment, and estimate the current aircraft position with the comprehensive consideration of the measurement data. This is the function of Kalman filter.

Then you must askHow can we estimate the aircraft position at this moment based on the estimated position of the aircraft at the previous moment?

answerKalman thinks that all state changes (position changes) are linear. What is linear? Last time the position was 0.3 and the speed was 0.2. So I estimate that the position for the next moment is 0.5. This is called linearity.

And then you’ll askWhat if not all state changes are linear? You’re like wind speed, it’s not linear.

answerCongratulations on your new algorithm. In fact, others have named this algorithm as extended Kalman filter. Now what we’re going to learn is Kalman filtering. All you need to remember is that Kalman filtering assumes that all changes are linear.

Now I know how to use the position of the aircraft at the last moment to estimate the position of the aircraft at the current moment, and I also know that the current position of the aircraft must be estimated based on the measured data received at the current moment. So how to consider it comprehensively? This involves a proportion. What is the proportion of these two data? This is the core essence of Kalman filter. Kalman filter algorithm should adjust this ratio dynamically. (there is a kind of golden mean tone, which believes not only in the measured data, but also in the estimated current moment position from the position of the last moment.)

A brief introduction to the process of Kalman filter

Each observation data, strictly speaking, should have a deviation value. For example, now that the thermometer measures 26 degrees and the deviation is 0.5 degrees, the real problem should be between (25.5,26.5), or written as $26pm0.5 $.

So wePredicted value, andObserved valuesPlus thisTwo separate deviationsA total of four known information to infer real, more essential data.

  • Predicted value: it can be calculated by pre-set formula and the real value at the last moment;
  • Observed value: read the value of measuring instrument directly.
  • Deviation of observed values: this can also be obtained directly;
  • Deviation of predicted value: This is calculated by the given formula from the deviation of the predicted value at the last time point.

In the following formula, the footmark K represents the time point and k-1 is the previous time point. The capital letters a, B and C are constants, which are set in advance; the capital letter H is a calculation.

  • Predicted value: $x ^ {forecast}_ k=A*x_ {k-1} ^ {real} + b * u_ {k-1}$
  • Observed value: $x ^ {observation}_ K$
  • Deviation of observed value: $p ^ {observation}_ K$
  • Deviation of predicted value: $p ^ {forecast}_ K = \ sqrt {(1-h) * (P ^ {forecast}_ {k-1})^2}$
  • Kalman gain H: $h_ K = \ frac {(P ^ {forecast}_ k) ^ 2} {(P ^ {forecast}_ k) ^ 2 + (P ^ {observation}_ k)^2}$
  • True value: $x ^ {true}_ k=H_ K * x ^ {observation}_ k+(1-H_ k) * x ^ {forecast}_ K$

It can be seen that the Kalman gain is a weighted average weight, whether the observation value is more important or the prediction value is more important; the importance of the two is determined by the deviation of the two, and the smaller the deviation is more important.

Of which $u_ The {k-1} $represents the control signal of the last time point, such as a robot. The state of the robot can feel the behavior of the robot itself, but in many cases, the control signal is not considered. For example, if the time series of stock market is filtered by Kalman filter, there is no control signal to control, just let it develop freely.

For example, the temperature of a room:

There are three moments in the morning, afternoon and evening (in fact, the interval between time points should be very short, here is just an example). The temperature in the morning is the observed value of 23 degrees, and the deviation is 0.5. Because the morning is the first time point, there is no predicted value;

In the afternoon, suppose a = 1, B = 0, so the predicted value in the afternoon is 23 degrees, and then the initial deviation is 1; the observation value in the afternoon is 25 degrees, and the deviation of the observation value is 0.5, so we can calculate the Kalman gain $h = – frac {1 ^ 2} {1 ^ 2 + 0.5 ^ 2} = 0.8 $, so the real value in the afternoon is $0.8 * 25 + (1-0.8) * 23 = 24.6$

In the evening, the predicted value at night is the real value of the previous time, so it is 24.6, and the deviation is $sqrt {(1-0.8) * 1} = 0.4472 $; the observation value at night is 20 degrees, and then the deviation is 0.5, so the Kalman gain gain is $h = – frac {0.4472 ^ 2} {0.4472 ^ 2} {0.4472 ^ 2} = 0.4444 ^ 2} = 0.4444 $, so the real value of this moment is $0.4444 * 20 + (1-0.4444) * 24.6 = 22.56$

In conclusion, what do we need? We need to know that the observation error is 0.5, and then the observation data at three time points: [23,25,20], and then after using Kalman filter, it becomes [23,24.6,22.56]. Similar to a smoothing effect.

How to implement Python?

from pykalman import KalmanFilter

def Kalman1D(observations,damping=1):
    # To return the smoothed time series data
    observation_covariance = damping
    initial_value_guess = observations[0]
    transition_matrix = 1
    transition_covariance = 0.1
    kf = KalmanFilter(
    pred_state, state_cov = kf.smooth(observations)
    return pred_state

The Kalman filter in pykalman library is used. Because the Kalman filter explained above is simplified and relies on the knowledge of normal distribution explained by the orthodox interpretation, the parameters of the Kalman filter here may not be corresponding to the Kalman formula given above one by one, which may cause some disconnection.

Here are the parameters:

  • initial_ state_ Mean and initial_ state_ Covariance: in the above formula, the initial value is the first observation value. However, in this method, the initial value is not the first observation value, but a value randomly sampled from a normal distribution. This normal distribution is called initial_ state_ Mean is the mean, and initial is used_ state_ Covariance is variance;
  • observation_ This can be equivalent to the observation deviation;
  • transition_ This is the prediction deviation;
  • transition_ Matrices is the capital letter A in the above formula, which is 1.

Operation results

From the above function, you can see the transition_ The covariance is 0.1, that is to say, the prediction deviation is 0.1. Therefore, assuming that the observation deviation is very small, it can be imagined that the result after the filter should be very close to the observation value. Here, the observation deviation is 0.001:1
Kalman filter Kalman filter of Xiaobai's AI
Then, assuming that the observation error is very large, we can think that the force of smoothing will be very large, and the result is:
Kalman filter Kalman filter of Xiaobai's AI
Finally, let’s take a look at the comparison chart before and after using Kalman filter to smooth data in a certain competition
Kalman filter Kalman filter of Xiaobai's AI

Kalman filter Kalman filter of Xiaobai's AI