Java – optimize code with thread pool and countdownlatch to improve execution efficiency

Time:2021-10-17

1. Problem description

The customer put forward a new requirement. After the development, it was found that querying the data within one hour took more time7Second, the customer must be dissatisfied. If he is dissatisfied, he must mention it to the leader. If the leader is unhappy, I will be deducted from my salary! So I want to use thread pool to optimize the code and improve the efficiency of the method.

2. Initial code

Click to view the code
//Query all sites
        QueryWrapper stationQW = new QueryWrapper<>();
        stationQW.lambda().eq(Station::getRegionCode, region);
        List stations = this.stationMapper.selectList(stationQW);
        List stationPOList = new ArrayList<>();
        long start = System.currentTimeMillis();
        for (Station station : stations) {
            long methodStart = System.currentTimeMillis();
            String stationCode = station.getStationCode();
            StationPO stationPO = new StationPO();
            BeanUtils.copyProperties(station, stationPO);
            //Total rainfall
            Float rainFall = stationDataMapper.queryRainFallByTime(startTime, endTime, stationCode);
            stationPO.setRainFall(rainFall);
            //Average temperature
            Float avgTemp = stationDataMapper.queryAvgTempByTime(startTime, endTime, stationCode);
            stationPO.setAvgTemp(avgTemp);
            //Maximum temperature
            Float maxTemp = stationDataMapper.queryMaxTempByTime(startTime, endTime, stationCode);
            stationPO.setMaxTemp(maxTemp);
            //Minimum temperature
            Float minTemp = stationDataMapper.queryMinTempByTime(startTime, endTime, stationCode);
            stationPO.setMinTemp(minTemp);
            //Maximum wind speed
            Float maxWind = stationDataMapper.queryMaxWindByTime(startTime, endTime, stationCode);
            stationPO.setMaxWind(maxWind);
            //Maximum wind speed
            Float enormousWind = stationDataMapper.queryEnormousWindByTime(startTime, endTime, stationCode);
            stationPO.setEnormousWind(enormousWind);
            //Average humidity
            Float avgHumidity = stationDataMapper.queryAvgHumidityByTime(startTime, endTime, stationCode);
            stationPO.setAvgHumidity(avgHumidity);
            long methodEnd = System.currentTimeMillis();
            System. Out. Println ("7 query times:" + new BigDecimal (methodend - methodstart). Divide (New BigDecimal (1000)). Doublevalue());
            stationPOList.add(stationPO);
        }
        long end = System.currentTimeMillis();
        System. Out. Println ("method time:" + new BigDecimal (end - start). Divide (New BigDecimal (1000)). Doublevalue());
The size of my site data collection is 37. There are 7 SQL statements in each cycle. The execution time of each SQL statement is about 0.8 seconds, and the time is wasted in the cycle. Therefore, imagine that the cycle creates a thread to execute the task. In this way, the total time is the time of a cycle.

3. Technology used

  • ThreadPoolExecutor thread pool
  • Countdownlatch lock
Here, let’s briefly talk about the countdownlatch lock. The function is that a thread will wait for other threads to execute before continuing. Specifically, it is implemented through a counter. The initial value of the counter is the number of threads. Every time a thread is executed, the counter value is – 1. When the counter value is 0, it means that all threads are executed, and then the threads waiting on the lock can resume work.

4. Overall thinking

First, create a thread pool, and then create a lock. Here, I directly set the size of the thread pool and the count of the lock to the size of the list, that is, the number of cycles, start the cycle, start the thread in the for cycle, execute the query data SQL of a site, and close a lock after the query is completed (countdown method). After waiting outside the loop for all threads to end (await method), close the thread pool (shutdown method) and end.

5. Optimized code

Click to view the code
//Query all sites
        QueryWrapper stationQW = new QueryWrapper<>();
        stationQW.lambda().eq(Station::getRegionCode, region);
        List stations = this.stationMapper.selectList(stationQW);
        List stationPOList = new ArrayList<>();
        ThreadPoolExecutor poolExecutor = ExecutorBuilder.create()
                . setcorepoolsize (stations. Size()) // initial thread
                . setmaxpoolsize (stations. Size()) // maximum threads
                . setworkqueue (New linkedblockingqueue < > (100)) // thread pool policy
                .build();
        CountDownLatch cdl = new CountDownLatch(stations.size());
        long start = System.currentTimeMillis();
        for (Station station : stations) {
            poolExecutor.execute(
                    () -> {
                        long methodStart = System.currentTimeMillis();
                        String stationCode = station.getStationCode();
                        StationPO stationPO = new StationPO();
                        BeanUtils.copyProperties(station, stationPO);
                        //Total rainfall
                        Float rainFall = stationDataMapper.queryRainFallByTime(startTime, endTime, stationCode);
                        stationPO.setRainFall(rainFall);
                        //Average temperature
                        Float avgTemp = stationDataMapper.queryAvgTempByTime(startTime, endTime, stationCode);
                        stationPO.setAvgTemp(avgTemp);
                        //Maximum temperature
                        Float maxTemp = stationDataMapper.queryMaxTempByTime(startTime, endTime, stationCode);
                        stationPO.setMaxTemp(maxTemp);
                        //Minimum temperature
                        Float minTemp = stationDataMapper.queryMinTempByTime(startTime, endTime, stationCode);
                        stationPO.setMinTemp(minTemp);
                        //Maximum wind speed
                        Float maxWind = stationDataMapper.queryMaxWindByTime(startTime, endTime, stationCode);
                        stationPO.setMaxWind(maxWind);
                        //Maximum wind speed
                        Float enormousWind = stationDataMapper.queryEnormousWindByTime(startTime, endTime, stationCode);
                        stationPO.setEnormousWind(enormousWind);
                        //Average humidity
                        Float avgHumidity = stationDataMapper.queryAvgHumidityByTime(startTime, endTime, stationCode);
                        stationPO.setAvgHumidity(avgHumidity);
                        long methodEnd = System.currentTimeMillis();
                        System. Out. Println ("7 query times:" + new BigDecimal (methodend - methodstart). Divide (New BigDecimal (1000)). Doublevalue());
                        stationPOList.add(stationPO);
                        //Lockout-1
                        cdl.countDown();
                    }
            );
        }
        try {
            //Wait for all threads to end
            cdl.await();
        } catch (InterruptedException e) {
            Staticlog. Error ("thread error: {}", e.getmessage());
        }
        poolExecutor.shutdown();
        long end = System.currentTimeMillis();
        System. Out. Println ("method time:" + new BigDecimal (end - start). Divide (New BigDecimal (1000)). Doublevalue());

6. Self summary

Because it is the first time to use multithreading, we have encountered many problems, although the final query time is shortened to0.7Seconds or so, but I don’t know whether it’s reasonable to use multithreading in this way. I have to continue to learn optimization later. Through this development, I also found the lack of my own knowledge reserve. Sure enough, learning java has to go from entry to soil. The final thing is the attitude and ideas when dealing with problems. Don’t be impatient. It will be more chaotic. It will always be solved slowly. Come on!

Recommended Today

SQL exercise 20 – Modeling & Reporting

This blog is used to review and sort out the common topic modeling architecture, analysis oriented architecture and integration topic reports in data warehouse. I have uploaded these reports to GitHub. If you are interested, you can have a lookAddress:https://github.com/nino-laiqiu/TiTanI recorded a relatively complete development process in my hexo blog deployed on GitHub. You can […]