Analyze Xu Song’s 175 songs and see why everyone likes listening to his songs so much

Time:2021-11-17

The text and pictures of this article come from the network, only for learning and communication, and do not have any commercial purpose. If you have any questions, please contact us in time for handling.

The following article comes from Luo Luopan, author Luo Luopan

Novices and Xiaobai who have just come into contact with Python can copy the following link to watch the basic introduction video of Python for free

https://v.douyu.com/author/y6AZ4jn9jwKW

 

preface

Do you still remember the broken bridge and snow? It’s an afternoon. Do you still remember to admit your mistake and order songs before class? Do you still remember Su Yan’s beautiful face at the same table

Recently, Xu Song held a concert and almost had the opportunity to listen to Xu Song’s songs and make this sharing. We can’t help but sigh that we are growing up slowly and can’t remember what we looked like in those years.

Analyze Xu Song's 175 songs and see why everyone likes listening to his songs so much

 

Web page analysis

Here, we enter the official website of Netease cloud music to find Xu Song’s personal single.

Analyze Xu Song's 175 songs and see why everyone likes listening to his songs so much

 

This is a post request with complex parameters, so selenium is used for crawling. In addition to obtaining basic information, it is necessary to obtain the ID of each song for lyrics crawling.

Lyrics crawler, we can use the following website to change different IDS, that is, the lyrics of different songs.

http://music.163.com/api/song/lyric?id=1309642305&lv=1&kv=1&tv=-1

 

Crawler code

The first is the crawler of all songs, using selenium. See the following for some codes.

from selenium import webdriver
from lxml import etree
import time
import csv

def get_info(url):
    driver = webdriver.Chrome()
    driver.maximize_window()
    driver.get(url)
    driver.implicitly_wait(10)
    iframe = driver.find_elements_by_tag_name('iframe')[0]
    driver.switch_to.frame(iframe)
    html = etree.HTML(driver.page_source)
    infos = html.xpath('//div[@class="srchsongst"]/div')    
    for info in infos:
        song_id = info.xpath('div[2]/div/div/a/@href')[0].split('=')[-1]
        song = info.xpath('div[2]/div/div/a/b/text()')[0]
        singer1 = info.xpath('div[4]/div/a')[0]
        singer = singer1.xpath('string(.)')
        album = info.xpath('div[5]/div/a/@title')[0]       
        print(song_id,song,singer,album)
        writer.writerow([song_id, song, singer, album])

if __name__ == '__main__':
      fp = open('music.csv', 'w', newline='', encoding='utf-8')
     writer = csv.
     writer(fp)
     writer.writerow(['song_id', 'song', 'singer', 'album'])
     url = 'https://music.163.com/#/search/m/?s=%E8%AE%B8%E5%B5%A9&type=1'
     get_info(url)

 

Analyze Xu Song's 175 songs and see why everyone likes listening to his songs so much

 

Some of these songs are cover and some are accompaniment. Before downloading the lyrics, we clean these data to ensure that they are original and not accompaniment by Xu Song.

def foo1(singer):    
If 'Xu Song' in Singer:
    return 'no cover'
else:
    return 'cover'

def foo2(song):
    If 'accompaniment' in Song:
        Return 'yes'
    else:
        Return 'no'

Music ['original '] = Music ['singer']. Apply (foo1)
Music ['style'] = Music ['song ']. Apply (foo2)

Music [(Music ['original'] = 'no cover') & (Music ['style'] = 'no')]. To_ csv('music_original.csv',index=None,encoding='utf-8'

 

Analyze Xu Song's 175 songs and see why everyone likes listening to his songs so much

 

Finally, we download the lyrics according to the previous URL.

import requests
import re
import json


url = ''

headers = {
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36'
}


def get_info(id):
    res = requests.get('http://music.163.com/api/song/lyric?id={}&lv=1&kv=1&tv=-1'.format(id),headers=headers)
    json_data = json.loads(res.text)
    lyric = json_data['lrc']['lyric']
    lyric = re.sub('\[.*\]','',lyric)
    return lyric

for i in range(music_original.shape[0]):
    try:
        fp = open('song/{}.txt'.format(music_original.iloc[i,1]),'w',encoding='utf-8')
        fp.write(get_info(music_original.iloc[i,0]))
        fp.close()
    except:
        pass

 

Analyze Xu Song's 175 songs and see why everyone likes listening to his songs so much

 

Data analysis

01 basic data

Xu Song has 175 songs in total, and is a well-established original and productive singer ~

02 album singles

Early Xu Song was an online singer, so they were all put in Xu Song’s single collection. Later, sugra had no bottom. Fog seeking enlightenment is a very good album.

Analyze Xu Song's 175 songs and see why everyone likes listening to his songs so much

 

03 lyrics cloud

Finally, the word cloud of Xu Song’s lyrics is made by using Jieba word segmentation.

Analyze Xu Song's 175 songs and see why everyone likes listening to his songs so much

 

Xu Song’s songs are sad and love, which can also be pruned through word clouds.