According to the weather forecast, TA’s city began to cool down

Time:2021-11-29

preface

The text and pictures of this article come from the network, only for learning and communication, and do not have any commercial purpose. If you have any questions, please contact us in time for handling.

PS: if you need Python learning materials, you can click the link below to get them by yourself

Python free learning materials and group communication solutions. Click to join

Basic environment configuration

  • python 3.6
  • pycharm
  • requests
  • parsel
  • csv

Relevant modules PIP can be installed

Determine target page

在这里插入图片描述

在这里插入图片描述

Analyze web pages

General operations of opening developer tools~
Take a look at the data returned from the web page. Do you have any information you want~
在这里插入图片描述
在这里插入图片描述
As shown in the figure, this website is a static web page, and the data is in the web pageTR taginside

Code part

  • Request web page data
import requests
url = 'https://weather.mipang.com/changsha/9yuefen.html'
headers = {
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Safari/537.36'
}
response = requests.get(url=url, headers=headers)
  • Parse the web page and extract the desired data
selector = parsel.Selector(response.text)
trs = selector.css('.tb tr')
for tr in trs:
    dit = {}
    date = tr.css('td:nth-child(1)::text').get()
    DIT ['date'] = date
    max_temperature = tr.css('td:nth-child(2)::text').get()
    DIT ['maximum temperature'] = max_ temperature
    min_temperature = tr.css('td:nth-child(3)::text').get()
    DIT ['minimum temperature'] = min_ temperature
    weather = tr.css('td:nth-child(4)::text').get()
    DIT ['weather'] = weather
    wind = tr.css('td:nth-child(5)::text').get()
    DIT ['wind direction'] = wind
    wind_power = tr.css('td:nth-child(6)::text').get()
    DIT ['wind'] = wind_ power
    print(dit)

在这里插入图片描述

  • Save data
F = open ('weather. CSV ', mode ='a', encoding ='utf-8-sig ', newline =' ')
csv_ Writer = CSV. Dictwriter (F, fieldnames = ['date', 'maximum temperature', 'minimum temperature', 'weather', 'wind direction', 'wind force'])
csv_writer.writeheader()
csv_writer.writerow(dit)
f.close()

在这里插入图片描述

Recommended Today

On the mutation mechanism of Clickhouse (with source code analysis)

Recently studied a bit of CH code.I found an interesting word, mutation.The word Google has the meaning of mutation, but more relevant articles translate this as “revision”. The previous article analyzed background_ pool_ Size parameter.This parameter is related to the background asynchronous worker pool merge.The asynchronous merge and mutation work in Clickhouse kernel is completed […]