What are the better dating software? Crawl three dating software reviews, terrible

Time:2021-8-14

The text and pictures of this article come from the network, only for learning and communication, and do not have any commercial purpose. If you have any questions, please contact us in time for handling.

The following article is from Python sophomore by Python sophomore

What are the better dating software? Crawl three dating software reviews, terrible

 

Taking 360 mobile assistant as an example, the address is: http://zhushou.360.cn/ , three popular dating software are selected, namely: Century Jiayuan, Lily marriage and love, and youyuan.com. We use Python to crawl the software review area to see the user’s evaluation.

Let’s take a look at the downloads and the proportion of good, medium and bad reviews of the three software (the unit in the figure below is 10000 times).

What are the better dating software? Crawl three dating software reviews, terrible

 

What are the better dating software? Crawl three dating software reviews, terrible

 

What are the better dating software? Crawl three dating software reviews, terrible

 

What are the better dating software? Crawl three dating software reviews, terrible

 

Let’s start crawling the comment area. Take century Jiayuan as an example. First, enter century Jiayuan in the search box to search, as shown in the figure:

What are the better dating software? Crawl three dating software reviews, terrible

 

Then, click the searched software to enter its details page, as shown in the figure:

What are the better dating software? Crawl three dating software reviews, terrible

 

Drop down the page to see the comment area, as shown in the figure:

What are the better dating software? Crawl three dating software reviews, terrible

 

At this time, open the developer tool and select the network item, click View more comments, and then you can see the getcomments request, as shown in the figure:

What are the better dating software? Crawl three dating software reviews, terrible

 

Through this request, we can dynamically obtain the data in the comment area. The parameter star is the starting comment index, and the parameter count is the number of comments loaded each time. Different applications can be specified through the parameters callback and Baike. The crawling code is implemented as follows:

headers = {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate, sdch",
    "Accept-Language": "zh-CN,zh;q=0.8",
    "Connection": "keep-alive",
    "Host": "comment.mobilem.360.cn",
    "User-Agent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.98 Safari/537.36 LBBROWSER"
}
def comment_spider(param, file_name):
    base_url = "http://comment.mobilem.360.cn/comment/getComments?c=message&a=getmessage&&count=50"
    start = 0
    for i in range(1, 50):
        Print ("page {}". Format (I))
        url = base_url + param + "&start=" + str(start)
        r = requests.get(url, headers=headers)
        data = re.findall("{\"errno\"(.*)\);}catch\(e\){}", r.text)
        #Convert to JSON format
        jdata = json.loads("{\"errno\"" + data[0])
        for message in jdata["data"]["messages"]:
            content = message["content"]
            print(content)
            with open(file_name + ".txt", "a", encoding="utf-8") as f:
                f.write(content)
        start = start + 50
        time.sleep(2)

 

We saved the crawled comment data to the txt file.

Next, we will show the word cloud of the comment data, and the code is as follows:

with open("yy.txt", "r", encoding="utf-8") as f:
    content = f.read()
    stylecloud.gen_stylecloud(text=content, max_words=600,
                              collocations=False,
                              font_path="SIMLI.TTF",
                              icon_name="fas fa-heart",
                              size=800,
                              output_name="yy.png")
    Image(filename="yy.png")

 

Finally, take a look at the user’s evaluation of the above software through the word cloud.

Jiayuan century:

What are the better dating software? Crawl three dating software reviews, terrible

 

Lily marriage:

What are the better dating software? Crawl three dating software reviews, terrible

 

Predestined network:

What are the better dating software? Crawl three dating software reviews, terrible