• Go one library a day, colly


    brief introduction collyIt is a powerful crawler framework written in go language. It provides a concise API, has strong performance, can automatically process Cookies & sessions, and provides a flexible extension mechanism. First, let’s introducecollyBasic concepts of. Then introduce it through several casescollyUsage and characteristics of:Pull GitHub treading, pull Baidu novel hot list, and download […]

  • On the “JS Crawler” from online blog to PDF e-book


    Recently, I was reading the source code analysis blog written by a big man. It’s not very interesting to watch it on the mobile phone during commuting, but there is no network card on the tablet, so I wanted to make an offline PDF version to facilitate learning and reading during commuting time. So here […]

  • Climbing novel


    Recently, I found a good-looking novel. Climb down and have a try. This script is not applicable to all web pages, but most web pages can still be used after a little modification according to the specific content #!/usr/bin/python # Author:Playon # -*- coding:utf-8 -*- # Time:2020/8/18 9:57 import requests from bs4 import BeautifulSoup import […]

  • Wechat website login authorization process is not clear, but also said that he was the front end of 3 years?


    The following is the development document from wechat The steps are shown in the figure: Step 1: the user agrees to authorize and obtains the code Step 2: exchange code for web access authorization_ token Step 3: refresh access_ token Step 4: pull user information Next, I will paste the code of the actual project, […]

  • Xi Da Pu Ben, wechat finally supports the external network to open the applet!


    preface After a long call, the wechat applet finally supports external arousal in the form of URL scheme. Long long ago ~ we can only use wechat development tag applet jump button in wechat web pagesOpen the applet, there is only such a single scene. In the actual business, we hope to open a small […]

  • Installing PHP in Ubuntu environment


    1、 InstallationInstall SSH sudo apt-get install ssh Install MySQL sudo apt-get install mysql-server Install Apache sudo apt-get install apache2 Install PHP (latest version) sudo apt-get install php Install the connection plug-in between PHP and Apache sudo apt-get isntall libapache2-mod-php Install phpMyAdmin sudo apt-get install phpmyadmin Install MySQL client sudo apt-get install mysql-client Install VIM sudo […]

  • Tips for “toppips” postman – you know, it’s very simple, but you don’t click in, you’ll never know


    One click Import request Import the asynchronous request of any web page into postman, open a web page, enter the network, and click XHR Right click to request copy – > copy as curl Enter postman, click Import and select rawtext Click continue Such a request is as like as two peas. The result is […]

  • Web standard web page structure


    Whether the background image or text size is on the page, it is obvious that there is a difference of one pixel. And I brought the bad problems in school. I was always so careless. After submitting the page for the first time, I had a lot of problems. Fortunately, my tutor pointed out to […]

  • CentOS 7 installing tinyproxy proxy server


    Mipu Technology, a start-up Internet company focusing on Internet Finance and big data mining, was officially incorporated in September 2016 and headquartered in Zhongguancun core functional area, Haidian District, Beijing. MIPO team,Members come from front-line Internet companies such as Baidu, Xiaomi, Ali and Innovation workshop, with rich experience in core technology research and development, with […]

  • JetBrains | the official JetBrains website cannot be accessed. The problem is solved


    The JetBrains official website cannot be accessed. The problem is solved Problem description Unable to access the JetBrains official website Maybe the specific web page display results are not like this. There are no pages anyway problem analysis Reason 1: the mapping of IP and domain name in the hosts file is incorrect This reason […]

  • Nginx solution 403 Forbidden


    The web page displays 403 Forbidden Nginx (Yum installation log is generally displayed in the / var / log / nginx / error. Log) error log open() “/web/www/one.txt” failed (13: Permission denied), client:, server: rumenz.com, request: “GET /one.txt HTTP/1.1”, host: “rumenz.com” Summarize four reasons: SELinux is not turned off Nginx startup user and working […]

  • Golang basic data structure and algorithm, web page ranking / PageRank, random walk


    origin Recently read < < my first algorithm book > > ([Japan] Ishida Baohui; Miyazaki Xiuyi)This series of notes is intended to use golang exercises Page ranking (PageRank / page ranking), random walk PageRank (also known as page ranking) is an algorithm that sorts search results when searching web pages. Web page ranking is an […]