• A more friendly scheme for automatic generation of front-end skeleton screen


    (horse comb technology original content, public number ID:mfwtech) According to a research report from Akamai, after interviewing 1048 online shoppers, we found that: About 47% of users expect their page to load in two seconds. If the page load time is more than 3S, about 40% of users will choose to leave or close the […]

  • Puppeteer pit under win2008 R2


    Address: Puppeteer pit under Win2008 R2 Do not use the default Windows command handler (CMD.exe) Use PowerShell or third-party command-line tools to execute Puppeteer, such as git bash that comes with Git. Perform various errors under cmd, start Chromium page and crash directly… Running Puppeteer under Win2008 is not only slow but also single-threaded. Clearly […]

  • CentOS 6.5-64 bit installation puppeteer, prompting Chrome can not start, find and install missing dependency packages – Lu Jiangmin Jingshang


    1. Enter the chrome directory cd /mnt/project/express/back/chrome-linux 2. Detecting missing dependency packages ldd chrome | grep not // Test results libatk-bridge-2.0.so.0 => not found libgtk-3.so.0 => not found libgdk-3.so.0 => not found libGL.so.1 => not found 3. Find out which software in the cloud contains the corresponding dependency packages below yum provides libatk-bridge-2.0.so.0 // result […]

  • How to avoid Puppeteer being detected by front-end JS


    Tools and information QQ group – Javascript advanced crawler – author self-built group, welcome to join! China Trademark Network Encryption Interface – Demonstration Only Awesome-java-crawler-tools and information about crawlers collected by the author Preface I started to look at puppeteer these two days and found that it could be detected by some number of front-end […]

  • Explain the use of proxy in Puppeteer


    background Proxies are sometimes required when using puppeteer to access pages. Environment: CentOS 7 Chrome installation Puppeteer uses chrome. Refer to chrome’s command line usage. I don’t see any instructions on the official website, but most execution files support help.Install chrome on the server: vim /etc/yum.repos.d/google-chrome.repo Adding content [google-chrome] name=google-chrome baseurl=http://dl.google.com/linux/chrome/rpm/stable/$basearch enabled=1 gpgcheck=1 gpgkey=https://dl-ssl.google.com/linux/linux_signing_key.pub Install […]

  • Hand-in-hand teaching you how to use Crawlab to build technology article aggregation platform (1)


    background Speaking of crawlers, most programmers think of popular frameworks like scrapy. Scrapy is really good, and it has a strong ecosystem and excellent visualization interfaces such as gerapy. However, there are still some things that can not be done, such as page flipping and clicking, mobile capture and so on. For these new requirements, […]