DWQA QuestionsCategory: Development Tool[help] java8 list < map < > > to filter out the problem of duplicate elements
Lingbo and tomorrow asked 1 month ago

Problem description
The data structure of list < Map > is as follows:

    List<Map<String, Object>> list = new ArrayList<>();
    
    Map<String, Object> map1 = new HashMap<>();
    map1.put("order_no", "123");
    map1.put("quantity", 10);
    map1.put("amount", 100);
    
    Map<String, Object> map2 = new HashMap<>();
    map2.put("order_no", "223");
    map2.put("quantity", 15);
    map2.put("amount", 150);
    
    Map<String, Object> map3 = new HashMap<>();
    map3.put("order_no", "123");
    map3.put("quantity", 5);
    map3.put("amount", 50);
    
    Map<String, Object> map4 = new HashMap<>();
    map4.put("order_no", "124");
    map4.put("quantity", 6);
    map4.put("amount", 60);
    
    Map<String, Object> map5 = new HashMap<>();
    map5.put("order_no", "223");
    map5.put("quantity", 7);
    map5.put("amount", 70);
    
    list.add(map1);
    list.add(map2);
    list.add(map3);
    list.add(map4);
    list.add(map5);

There is a requirement to judge whether there is any in the list < Map > above Map.key=order_ No, its value is repeated, and the repeated items are taken out. As shown in the example, the order should be caught in the end_ No = 123223, I write these two orders as follows:

//Defining a transition List2 is consistent with list
    List<Map<String, Object>> list2 = new ArrayList<>();
    list2.addAll(list);
    
    List<Map<String, Object>> collect = list.stream().filter(x->{
        long count = list2.stream().filter(x2->x2.get("order_no").equals(x.get("order_no"))).count();
        If (count > 1) {// judge whether it is repeated
            return true;
        }
        return false;
    }).collect(Collectors.groupingBy(x->x.get("order_no"))).entrySet().stream().map(x->{
        Map<String, Object> tmp = new HashMap<>();
        tmp.put("key_order", x.getKey());
        tmp.put("order_list", x.getValue());
        Return TMP; // display duplicate order data in groups
    }).collect(Collectors.toList());

At present, although the function is realized, considering the order quantity of tens of thousands or even more, the same transition is redefined. The writing method of list is rough and the efficiency is not high. Do you have a more concise, efficient and elegant way to realize the function?

3 Answers
Spicy chicken answered 1 month ago

Can’t you just use set to check for repetition? It’s just a loop. You’re too scary

        Set<String> set = new HashSet<>();
        Map<String,List<Map<String,Object>>> valMap = new HashMap<>();
        for(Map<String,Object> item:list){
            String id = item.get("order_no").toString();
            if(set.contains(id)){
                List<Map<String, Object>> l = valMap.computeIfAbsent(id, k -> new ArrayList<>());
                l.add(item);
            }
            set.add(id);
        }

        for(Map.Entry<String,List<Map<String,Object>>> entry:valMap.entrySet()){
            System.out.println(JSON.toJSONString(entry.getValue()));
        }
        // print
        // [{"order_no":"123","amount":50,"quantity":5}]
        // [{"order_no":"223","amount":70,"quantity":7}]
Lingbo and tomorrow replied 1 month ago

Hello, I have considered the set method, but since other keys such as quantity in each map are not the same, the same order number is not filtered. Before writing, the following is:List<Map<String, Object>> list3 = new ArrayList<>(new HashSet<>(list));

Spicy chicken replied 1 month ago

I wrote it down to see if it’s what you want

Du Shixiang answered 1 month ago

Java8 stream itself provides the key based de duplication.distinct()Method, but there is no way to weigh according to value, we have to write an extension to him.

import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.function.Function;
import java.util.function.Predicate;

/**
 *@ author fan Xian. Created in 2018 / 12 / 01 00:01
 */
public class StreamEx {

    public static <T> Predicate<T> distinctByKey(Function<? super T, Object> keyExtractor) {
        Map<Object, Boolean> seen = new ConcurrentHashMap<>();
        return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;
    }
}

test

list
                .stream()
                .filter(StreamEx.distinctByKey(x -> x.get("order_no")))
                .forEach(x -> {
                    System.out.println(x.toString());
                });

//        {order_no=123, amount=100, quantity=10}
//        {order_no=223, amount=150, quantity=15}
//        {order_no=124, amount=60, quantity=6}

It can be seen that the above code prints the data information after de duplication.
You only need the

return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) == null;

Change to

return t -> seen.putIfAbsent(keyExtractor.apply(t), Boolean.TRUE) != null;

Can meet your requirements.

Beautiful code

INF_ ruthless replied 1 month ago

Great

asd1245dss answered 1 month ago

This is simple. When adding elements to the list, a map is used to index, and the key is order_ No, value is the map of the element. Because there is only one reference, there will be no big space problem. At the same time, the O (1) query ability of map can be used