When a target site detects abnormal user access to the site (frequency and duration of operation detected and proxy IP detected), it usually:
1. Mask the IP address
2. Display misleading information to the IP address
3. Slow down the response time
Target sites log the IP addresses of visitors and analyze the activity of those IP addresses. Assuming you are using a traditional data center proxy or a low-anonymity proxy, the target site can do the following:
The ability to identify activity (request rate) from a single IP in a given period of time is much higher than the human ability
Identify IP addresses from the list of proxy servers that these target sites can access
Identify these IP addresses as having the same subnet mask range
How to avoid detection?
To avoid detection due to the number of requests per IP, you can reduce the number of requests per second. However, this can also slow down your data fetching speed. In order to avoid being detected by the target website because your IP came from a proxy server, you must cycle your requests through different residential IP addresses. You should be able to loop through enough ips to ensure that the target site cannot detect your activity. With residential IP, there is no subnet blocking range, residential IP proxy with a legitimate IP address, will not blacklist you from the site. In the case of a data center proxy, the site owner can detect that it belongs to the data center and not the ISP.
