On the set get of redis

Time:2021-12-28

What is the bloom filter

This (reference address) is used in Java projects. Let’s talk about it today

The name is the same as every law. You ask why it is called Newton’s law, because it was invented or discovered by Newton.

What can he do? It maps a binary vector to a function. Bloom filter can be used to detect whether there is a set of elements or for fast retrieval.

Disadvantages: there are certain deletion problems and error recognition rate

Advantages: query time and space are far more than ordinary algorithms

How is bloom filter implemented

When adding an item or element, create a hash function and a key to form a mapping. The set data = 1. As long as you judge that = 1 during retrieval, you will know whether the data exists. With this method, if 0 is found during query, it must prove that it does not exist. Conversely, if 1 proves that the element is likely to exist,

Note why it is said here that it is likely to exist because it has a certain identification error, but this error can be ignored in the actual production process. After all, do the advantages outweigh the disadvantages.

I’m dizzy when I read the text. I draw pictures without moving. If I look at it, I should understand a lot.

On the set get of redis

Say something reasonable.

What on earth can bloom filter do?

Not to mention the special ID, the database ID is basically self increasing, right! We pass the ID back end to the DB for query, which is very reasonable.

But what if we use negative numbers? One or two doesn’t matter. What if thousands? Basically, the database will be under a lot of pressure, and the server configuration is not to mention for the time being. It is possible to slow down the system operation speed or even downtime. Is this a bit incisive of the bloom filter. [dog head]

Such hanging also has a price, because it is also uncertain, and there is a certain degree of misjudgment!

Q: why is it misjudged?

A: the searched keys are not in the container, but the keys obtained after hash are all 1. If there is a blacklist in the bloom filter, you can directly create a white list.

Q: why is it not easy to delete?

A: we mentioned that the correct data key value = 1, but we can’t delete it because = 0, which may affect the judgment of other elements. However, you can learn about the “next article” of counting bloom filter

How did you achieve it after saying so much

1: Estimated quantity n and expected misjudgment rate FPP

2: Size of hash function and bit set

Bit set size

Function hash selection, the estimated value N and bit array length m obtain the hash function key

How to use it? Add to Maven project

 <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>23.0</version>
 </dependency>    

A test code I wrote

/**
 *Bloom filter - used for redis cache penetration
public class TestBloomFilterByDZZ {

    private static int total = 19999;
    private static BloomFilter<Integer> bfilter = BloomFilter.create(Funnels.integerFunnel(), total);

  //Initialization data
    public static void main(String[] args) {
        for (int i = 0; i < total; i++) {
            bfilter.put(i);
        }

        //Is there anything that doesn't match
        for (int i = 0; i < total; i++) {
            if (!bfilter.mightContain(i)) {
                System. out. Println ("did you pay attention to the slip of Northwest zongzi...);
            }
        }

        //How many matches are there in no longer
        int count = 0;
        for (int i = total; i < total + 10000; i++) {
            if (bfilter.mightContain(i)) {
                count++;
            }
        }
        System. out. Println ("cannon fodder running:" + count);
    }

}

Applicable business scenarios

1: When there is a large amount of warehousing data, it can be used as a name or unique piece for inspection. If it exists, it will be skipped. If it does not exist, it will be warehousing

2: Filter spam. This is a calculation. You can understand it in combination with your own business.

Daily attention, what can be left when being tortured by business.

Recommended Today

Handwritten Vue2 series compiler

When learning becomes a habit, knowledge becomes common sense.thank you allfocus on、like、collectandComment。 New videos and articles will be sent on the WeChat public account as soon as possible, please pay attention:Li Yongning lyn Articles have been included ingithub repository liyongning/blog, Welcome to Watch and Star. foreword Next, we will officially enter the handwritten Vue2 series. […]