Implementation of Python method to quickly verify whether proxy IP is effective

Time:2021-11-29

Sometimes, we need to use proxy IP, for example, when crawling, but after we get the IP, we may not know how to verify whether these IP are valid. At this time, we can use Python to carry the IP to simulate accessing a website. If we fail to access it for many times, it means that the proxy is invalid.

The code is as follows:

import requests
import random
import time

http_ip = [
    '118.163.13.200:8080',
    '222.223.182.66:8000',
    '51.158.186.242:8811',
    '171.37.79.129:9797',
    '139.255.123.194:4550'
]

for i in range(10):
    try:
        ip_proxy = random.choice(http_ip)
        proxy_ip = {
            'http': ip_proxy,
            'https': ip_proxy,
        }
        Print ('Ip using proxy: ', proxy_ip)
        response = requests.get("http://httpbin.org/ip", proxies=proxy_ip).text
        print(response)
        Print ('current IP valid ')
        time.sleep(2)
    except Exception as e:
        print(e.args[0])
        Print ('current IP is invalid ')
        continue

The operation results are as follows:

IP using proxy: {HTTP ‘:’118.163.13.200:8080′,’https’:’118.163.13.200:8080 ‘}
HTTPConnectionPool(host=’118.163.13.200′, port=8080): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError(‘Cannot connect to proxy.’, NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x00000247674F5F88>: Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connecting party did not reply correctly after a period of time or the connected host did not respond. ‘))
The current IP is invalid
IP using proxy: {HTTP ‘:’51.158.186.242:8811′,’https’:’51.158.186.242:8811 ‘}
{
  “origin”: “51.158.186.242”
}

Current IP valid
IP using proxy: {HTTP ‘:’222.223.182.66:8000′,’https’:’222.223.182.66:8000 ‘}
{
  “origin”: “139.202.62.84, 222.223.182.66”
}

Current IP valid
IP using proxy: {HTTP ‘:’51.158.186.242:8811′,’https’:’51.158.186.242:8811 ‘}
{
  “origin”: “51.158.186.242”
}

Current IP valid
IP using proxy: {HTTP ‘:’51.158.186.242:8811′,’https’:’51.158.186.242:8811 ‘}
{
  “origin”: “51.158.186.242”
}

Current IP valid
IP using proxy: {HTTP ‘:’222.223.182.66:8000′,’https’:’222.223.182.66:8000 ‘}
HTTPConnectionPool(host=’222.223.182.66′, port=8000): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError(‘Cannot connect to proxy.’, NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x00000247675067C8>: Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connecting party did not reply correctly after a period of time or the connected host did not respond. ‘))
The current IP is invalid
IP using proxy: {HTTP ‘:’139.255.123.194:4550′,’https’:’139.255.123.194:4550 ‘}
HTTPConnectionPool(host=’139.255.123.194′, port=4550): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError(‘Cannot connect to proxy.’, NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x00000247674F55C8>: Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connecting party did not reply correctly after a period of time or the connected host did not respond. ‘))
The current IP is invalid
IP using proxy: {HTTP ‘:’51.158.186.242:8811′,’https’:’51.158.186.242:8811 ‘}
{
  “origin”: “51.158.186.242”
}

Current IP valid
IP using proxy: {HTTP ‘:’51.158.186.242:8811′,’https’:’51.158.186.242:8811 ‘}
{
  “origin”: “51.158.186.242”
}

Current IP valid
IP using proxy: {HTTP ‘:’222.223.182.66:8000′,’https’:’222.223.182.66:8000 ‘}
HTTPConnectionPool(host=’222.223.182.66′, port=8000): Max retries exceeded with url: http://httpbin.org/ip (Caused by ProxyError(‘Cannot connect to proxy.’, NewConnectionError(‘<urllib3.connection.HTTPConnection object at 0x0000024767514908>: Failed to establish a new connection: [WinError 10060] The connection attempt failed because the connecting party did not reply correctly after a period of time or the connected host did not respond. ‘))
The current IP is invalid

This is the end of this article on the implementation of Python’s method to quickly verify whether the proxy IP is valid. For more information about Python’s verification of whether the proxy IP is valid, please search the previous articles of developeppaer or continue to browse the relevant articles below. I hope you will support developeppaer in the future!

Recommended Today

On the mutation mechanism of Clickhouse (with source code analysis)

Recently studied a bit of CH code.I found an interesting word, mutation.The word Google has the meaning of mutation, but more relevant articles translate this as “revision”. The previous article analyzed background_ pool_ Size parameter.This parameter is related to the background asynchronous worker pool merge.The asynchronous merge and mutation work in Clickhouse kernel is completed […]