Table of Contents

BOT Protection

Rama Sadhu Updated by Rama Sadhu

This page gives insights into the total no. of BOT attacks, various protection methods, BOT trends, the correlation of the risk scores from various modules - Allow Good Bots/ Block Good Bot Pretenders, Tor IP, User Agent Based Detection, Suspicious Countries, IP Reputation, Data Center IP, and so on. 

Tolerance Status 

  • The tolerance status is defined as the act of allowing bot attacks at a certain level.  
  • When the risk score of a host IP against a website breaches its tolerance level, then the IP address of the website will automatically be blocked for 10minutes.
During the blocking period, if a valid request is received from the same host IP address, it will also be blocked in the TTL period. However, for a legitimate request, the TTL will not be refreshed.

Parameter 

Description 

High (relaxed) 

High tolerance means the customer is willing to accept the bot attacks. 

Risk score is 80 

Medium 

Medim tolerance means the customer is accepting the bot attacks by any chance. 

Risk Score is 70 

Low (paranoid) 

Low tolerance means the customer is not compromising to accept bot attacks.  

Risk score is 60 

Log Only 

When the tolerance level is breached no attacks will be blocked but they will be logged. 

Log and Block 

When the tolerance level is breached, it blocks further bot attacks, even the valid requests will be blocked till the TTL time is elapsed. 

By default, the risk tolerance level is set as high for newly onboarding websites. 
  • Click the drop-down to change the tolerance status and then click the Update button.

 

Total Requests 

The Bot protection dashboard shows the number of Bot requests out of total requests along with their bandwidth usage for a selected time span. 

Users can change the time span to Last 24 Hours, Last 7 Days, and Last 7 Days. 

BOT Classification Trend 
  • The following trend displays the count of total bot attacks, attacks blocked, and allowed for each module. 
  • Users can change the time span to Last 24 Hours, Last 7 Days, and Last 7 Days. 
The Blue color indicates the total number of attacks, the Black color indicates the attacks which are blocked, and the Green color indicates the attacks which are allowed.

Traffic Graph 

The graph displays the comparison between the total traffic and the bot traffic for a selected time span. 

The Blue color indicates the total traffic and the black color indicates the bot traffic.

BOT Classification Table

The following table helps users to gain a deeper understanding of the bot classification, the total bot requests received, and the total requests blocked and allowed.  

 Click Analyze to see the full bot details of a website.

Top 20 URIs Accessed by Bots 
  1. The table provides a quick summary of the top 20 URI data accessed by BOTs with the respective count.  
  2. Click Analyze tab to see the further details of URI data. 

Top 20 User Agents 
  1. Select UserAgents from the drop-down menu. 
  2. User agents are softwares’ that retrieve web content for end users. The table provides the user agents data from which bot requests are received and their respective counts. 
  3. Click Analyze to see further details of user agents used by bots.
Top 10 Country Data
  1. Select Countries from the drop-down. 
  2. The number of bot attacks by the top 10 countries is displayed. Click Analyze to view further details. 

 

Policies

The following tables consists of different policies which help users to configure based on the requirements.

  1. Allow Good Bots/ Block Good Bot pretenders 
  •  By default, the policy is available in activated mode. 
  • When policy is in activated mode, it automatically allows the good bots and blocks the good bot pretenders 
  • Click the Settings icon to configure the policy. Two types of configurations are available.
  • Standard Configuration: - This option allows the list of standard bots. Users can not remove certain bots. 
  • Custom Configuration: - There are 3 options given under custom configuration. 

- Google 

- Bing 

-DuckDuckGo 

  • By default, all the 3 options selected in custom configuration. Undo the checkmark to remove certain bots. 
The maximum risk score threshold for Good Bot Pretenders is 100. 
  1. TOR IP 
  •  By default, the policy is available in activated mode. 
  • If a user selects to block TOR IPs (from settings) and a request comes from the IP which is identified as TOR, then the risk score will be given as 100.  
  • If a user allows TOR IPs and a suspicious request is found, then the risk score will be increased by 20. 
  • Click the Settings icon to block the TOR IPs.
  • Click on the checkmark to block all the TOR IPs and then click Save to apply changes. 
The maximum risk score threshold for TOR IPs is 30. 
  1. User Agent Based Detection 
  • By default, the policy is available in activated mode. 
  • User Agents helps to determine the nature of a request and based on that risk score will be given.  
  • We have a database for user agents. If a request is detected from any of the user agents in data base, it automatically blocks. 
  1. Suspicious Countries 
  • This policy helps to mark a country as suspicious instead of blacklisting the country. So, a request that comes from a suspicious country is used to determine the risk score of requests. 
  • Click the Settings icon on the right side to mark a country as suspicious. 
  • Click Add Country+ to add a country as suspicious.  
  • Click Delete Icon on the right side of the country to remove the country from blocklist. 

  • When a request comes from the blocked suspicious country, the score will be given as 20. 
  • A country will be blocked only when the whole threshold value (attacks by bot pretenders, reputed Ips, and so on, the cumulative risk score) of the website is breached. 
  • Click the checkbox to block all the listed suspicious countries in the list and then click Save to apply changes. 
  1. IP Reputation 
  •  We have a database to identify the malicious IPs. If any malicious activity is identified from this IP, the risk score will be increased. When the risk score breaches the website threshold, the IP will be blocked automatically. 
  1. Data Center IP 
  •  We have a database for Data center IPs. The requests coming from Data center are mostly Bots. 
  • If we identify attacks from data centers/servers, the policy will block the server or data center automatically. 
  1. Anomaly Behavior Detection
  •  The policy works based on the cumulative attacks log data. 
  • If any attack is blocked by BOT module(TOR IPs, Good Bot pretenders, IP Reputation, and so on) or by Behavioral DDOS policy, then the risk score will not be calculated under anomaly behavior detection and then the policy will not be triggered.  
  •  Irrespective of the type of attack (like Injection attack, Local file inclusion, cross-site scripting attack, and so on) on website, risk score will be calculated based on the attacks count and attacks severity. 
  • The attacks which are blocked by using core rules and custom rules, based on the severity of attack risk score will be calculated and the IP will be blocked once the risk tolerance level is breached.

 

Risk Score for Critical Severity– 10, Risk Score for High Severity – 5, Risk score for Medium Severity – 3 

 

  • When the risk score breaches the website threshold tolerance value, then the IP will be blocked for 10 min (TTL). 
  • During the blocking period, if an attack or a legitimate request is received from the same host IP address, that IP address will be blocked in the TTL period, but the TTL will not be refreshed.
  • Once the TTL is completed, again the risk score will be calculated, and then the policy will be triggered based on the tolerance level. 
The maximum risk score threshold for anomaly detection is 100. 
Anomaly Behavior Detection policy is now only available for Premium and API plan customers.

How Does Anomaly Behavior detection work? 

Walk through the following example for a better understanding. 

As per the tolerance level IP will be blocked only if the risk score is >60. 

How did we do?

Asset Discovery

Browser Protection

Contact

This site is protected by hCaptcha and its Privacy Policy and Terms of Service apply.