Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

Time:2021-7-5

The text and pictures of this article are from the Internet, only for learning and communication, and do not have any commercial use. If you have any questions, please contact us in time.

The following article comes from Zen of Python data analysis, written by little dull bird

Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

 

preface

Today I would like to share with you a very practical article, using Folium to make the distribution map of traffic electronic eyes in Beijing. I am no longer afraid of being photographed by hidden cameras

The results are as follows:

Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

 

It is mainly divided into the following two steps:

1. Electronic eye geographic data acquisition

The website is as follows:

https://www.icauto.com.cn/weizhang/wzd/110000/

 

Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

 

This website can obtain the electronic eye data of various regions in China

Taking Beijing as an example, we use Python to crawl the names of all electronic eyes in Beijing. The code is as follows:

url1='https://www.icauto.com.cn/weizhang/wzd/110000/list_1.html'
response=requests.get(url1,headers=header)
soup=BeautifulSoup(response.text,'html.parser')
results=soup.find('div',class_='cdz-ccotent').find_all('li')
for result in results:
    time.sleep(0.5)
    name=result.find('a').text
    index=result.find('span').text.split(':')[1]

 

After packet capture analysis, we can get the interface to obtain the longitude and latitude information according to the name of the electronic eye

https://api.map.baidu.com/?qt=gc&wd= Zhongba tunnel section of Beijing West Road & CN = Beijing & ie = UTF-8 & oue = 1 & fromproduct = jsapi & res = API & AK = s8ss5dbsz7blri3bcvrayaymanqlxoyeo

 

The results are as follows:

Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

 

You mainly look at the coord parameter. Just now, I was a little confused. I don’t know what it is. I checked it for a long time and found out that this is the Mercator plane coordinates used by Baidu map. Using Baidu map API, it can be converted into longitude and latitude. The conversion code is as follows:

url3=' http://api.map.baidu.com/geoconv/v1/?coords= {} & from = 6 & to = 5 & AK = change to your AK '. Format (STR (coord ['x'] + ',' + str (coord ['y '])
response=requests.get(url3,headers=header)
result=json.loads(response.text)['result'][0]
lon=result['x']
lat=result['y']

 

Finally, save the result to the CSV table

with open('dianziyan.csv', 'a+', newline='', encoding='gb18030') as f:
    f_csv = csv.writer(f)
    f_csv.writerow([name, index, lon,lat])
Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

 

2. Punctuation with Folium

The method of Folium map punctuation has been introduced before

Generation of parking distribution map of Guangzhou by Python

The code is as follows:

import pandas as pd
data=pd.read_csv('dianziyan.csv',encoding='gbk')
import folium
from folium import plugins
Camera_ Map = Folium. Map (location = [data ['latitude ']. Mean(), data ['longitude']. Mean(), zoom_ start=10,control_ scale=True,)
incidents = folium.map.FeatureGroup()
for name,row in data.iterrows():
    incidents.add_child(
        Folium. Circlemarker (# circlemarker means flower circle)
            [row ["latitude"], row ["longitude"], coordinates of each parking lot
            Radius = 7, # circle radius
            Color ='yellow ', # the outer ring color of the logo
            Fill = true, # fill
            fill_ Color = red ', fill color
            fill_ Opacity = 0.4, # fill transparency
            
        )
    )

Camera_map.add_child(incidents)
Camera_map.save('Camera_map1.html')

 

Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

 

After zooming in, it can be found that the coordinate positioning is not very accurate, some seriously deviate from the route, and some even locate in the lake, because the base map coordinates and electronic eye longitude and latitude standards are not unified

In order to locate accurately, we unified the base map and electronic eye longitude and latitude as Gaode map

The conversion function of Baidu longitude and latitude to Gaode longitude and latitude is as follows:

import math
def bdToGaoDe(lat,lon):
    """
    Baidu longitude and latitude to Gaode longitude and latitude
    :param lon:
    :param lat:
    :return:
    """
    PI = 3.14159265358979324 * 3000.0 / 180.0
    x = lon - 0.0065
    y = lat - 0.006
    z = math.sqrt(x * x + y * y) - 0.00002 * math.sin(y * PI)
    theta = math.atan2(y, x) - 0.000003 * math.cos(x * PI)
    lon = z * math.cos(theta)
    lat = z * math.sin(theta)
    return lat,lon

 

Read the data and convert it

import pandas as pd
data=pd.read_csv('dianziyan.csv',encoding='gbk')
for name,row in data.iterrows():
    Print (bdtogaode (row ["latitude"], row ["longitude"])

 

Change the base map to Gaud map, and then convert the punctuation to electronic eye icon:

import folium
from folium import plugins
Camera_ Map = Folium. Map (location = [data ['latitude ']. Mean(), data ['longitude']. Mean(), zoom_ start=10,zoom_ control='False',
                     tiles='http://webrd02.is.autonavi.com/appmaptile?lang=zh_cn&size=1&scale=1&style=7&x={x}&y={y}&z={z}',attr='AutoNavi')
incidents = folium.map.FeatureGroup()
Tooltip: 'please click me to view the information of this point'
for name,row in data.iterrows():
    incidents.add_child(
        Folium. Marker (# circle marker means flower circle
            [bdtogaode (row ["latitude"], row ["longitude"] [0], bdtogaode (row ["latitude"], row ["longitude"] [1]], the coordinates of each parking lot
            icon=folium.Icon(color='green', prefix='fa', icon='bullseye')
        )
    )

Camera_map.add_child(incidents)
Camera_map.save('Camera_map2.html')

 

Python crawls all the electronic eye names in Beijing, and is no longer afraid of being secretly photographed by hidden cameras

 

Through tiles, you can set different map tiles. Here, it is set as Gaud map tiles

The folium. Icon class can set color and icon_ Color, icon, angle and prefix are five parameters

The color options include: ['Red ',' Blue ',' green ',' Purple ',' Orange ',' darkred ',' lightred ',' Bee ',' darkblue ',' darkgreen ',' cadetblue ',' darkpurple ',' white ',' Pink ',' lightblue ',' lightgreen ',' gray ',' Black ',' lightgray '] or HTML color code
icon_ Same as above
Icon can find the corresponding name in the font awesome website, and set the prefix parameter to 'FA'
Angle is set in degrees

 

This paper only shows the location of the electronic eye in Beijing. Interested friends can make a thorough study and make a map of the distribution of the national traffic electronic eye.