With data provided by the bot traffic management module, bot traffic analysis quickly analyzes the bot impacts on domain names in terms of bot feature metrics, including the types of bots, proportion of actions, bot score distribution, top data by request count and URLs that may be affected. You can click View details to view the bot details of an access source, and its access characteristics and exceptions detected.
The bot traffic details section displays the bot traffic details of top 10 access sources. You can also view their session access information and logs if session settings are configured, and the information of these access sources/IPs by retrieval.
Prerequisites
You have subscribed to the bot traffic management service and enabled bot traffic analysis.
Directions
1. Log in to the WAF console and select Bot Traffic Analysis on the left sidebar. Open the Bot traffic details tab.
2. On the page displayed, click the drop-down list in the upper-left corner and select a domain name.
3. Specify a date or use the filter to search top 10 access sources of all domain names or a specific domain name. Then click View logs of the access source you want to view.
4. To view traffic details of an access source, click View details on the right.
Viewing access overview
The bot traffic details page shows you the estimated risk value of the access source and the information of the hit policy, and allows you to take measures such as adding the access source to the allowlist/blocklist and creating custom rules targeting the access source.
Field description:
IP address and tag: It displays the IP address of an access source and the hit tag that identifies the bot category.
Last request: It displays the score of the last access request from the access source and the risk level.
Session count: It displays the number of continuous sessions on the website accessed occurred in the last access request.
Access address: It displays the domain name of the access source.
Exception feature: It displays modules that detect exceptional features of the access source.
Hit modules: It displays modules that take actions to combat bots.
Policy ID: It is the ID of a hit policy.
View access logs: It redirects you to the access logs page where you can view the access details of the access source.
Add to allowlist: It allows you to add the access source to the allowlist.
Add to blocklist: It allows you to add the access source to the blocklist.
Add custom rules: It allows you to add custom rules targeting the access source.
Viewing bot scores
In the bot score distribution and bot action distribution sections, you can view the distribution of bot scores and bot actions within the selected period, helping you determine the risk level of the access source.
Viewing bot information
The bot traffic details page also displays information of the automated access source including the features of the bot and access request, threat intelligence and AI evaluation information, bot flow statistics and sessions. Using this data, you can quickly identify exceptions in the access request and take measures against the bot.
Basic session information
On the basic session information tab, you can view the information of the access source IP and session separately.
Field description:
IP information
Access source IP: IP address of an access source.
City: City where the access source is located.
Region: Country where the access source is located.
IP type: IP type of the access source.
IP owner: IP owner of the access source.
Session information
Session average speed: The average session speed of the access source in the latest session, which is calculated by the total number of session requests/session duration. Unit: times/minute.
Total sessions: The total number of sessions of the access source in the latest session.
Whether Robots.txt exists: Whether the access source has accessed the Robots.txt file, which is often accessed by bot sessions.
Session duration: Amount of time the latest session initiated by the access source.
Request feature information
On the request feature information tab, you can view the information of the request features, Cookie, User-Agent, Referer, and Query in the session request.
Field description:
Type
Metric
Description
Request feature information
Percentage of repeated URLs
Percentage of the repeated URLs in a session request. The value range is 0-1. Set this parameter based on your actual business needs. A value that is too high or too low suggests an exception (which must be determined based on the actual business conditions).
Total URL types
Number of deduplicated URLs in a session request.
Minimum URL depth
Minimum directory levels of URLs in a session request.
Maximum URL depth
Maximum directory levels of URLs in a session request.
Average URL depth
Average directory levels of URLs in a session request.
Total URLs
Total number of URLs visited in a session request (including duplicates).
Cookie information
Whether Cookie is abused
Different types of UAs use the same Cookie.
Cookie exist
Whether the Cookie exists in all session requests.
Percentage of repeated Cookies
Percentage of the repeated Cookies in a session request. The value ranges from 0-1.
Cookie validity
Percentage of the Cookies that can be parsed in a session request.
Most used Cookie
Most used Cookies in a session request.
Percentage of the most used Cookies
Percentage of the most used Cookies in a session request.
User-Agent information
User-Agent type
User-Agent type of the request user in a session request.
User-Agent exist
Whether User-Agent exists in a session request.
User-Agent randomness index
Random distribution of User-Agents in a session request. When the reference value threshold exceeds 0.6, an exception is suspected; when it exceeds 0.92, an exception is basically confirmed.
User-Agent type
Number of deduplicated User-Agents in a session request, which is valid only for non-proxy IPs. A value that is too high suggests an exception (which must be determined based on the actual business conditions).
User-Agent existence rate
Existence rate of UAs in a session request, which ranges from 0 to 1. A value too small may be suspected as an exception (which must be determined based on the actual business conditions).
Most used User-Agent
Most used value of the HTTP User-Agent in a session request.
Percentage of the most used User-Agents
Percentage of the most used HTTP User-Agent values in a session request.
User-Agent similarity rate
Similarity between the most used value and the rest in a session request.
Referer information
Percentage of repeated Referers
Percentage of repeated Referers in a session request, which is valid only for access through browsers and ranges from 0 to 1. A value that is too high suggests an exception (which must be determined based on the actual business conditions).
Referer exist
Existence rate of Referers in a session request, which ranges from 0 to 1. A value too small may be suspected as an exception. It's available for browser access.
Referer existence rate
Existence rate of Referers in a session request, which ranges from 0 to 1. A value too small may be suspected as an exception. It's available for browser access.
Whether Referer is abused
Different types of UAs use the same Referer.
Most used Referer
Most used value of the HTTP Referer in a session request.
Percentage of the most used Referers
Percentage of the most used HTTP Referer values in a session request.
Query information
Percentage of repeated Query parameters
Percentage of repeated GET request parameters (`Query` content) or POST request parameters (`Body` content) in a session request, which ranges from 0 to 1. Set this parameter based on your actual business needs. A value that is too high or too low suggests an exception (which must be determined based on the actual business conditions).
Total query parameter types
Most used parameters in a session request, which may be GET request parameters (`Query` content) or POST request parameters (`Body` content).
Threat intelligence
The threat intelligence tab displays the known information that matches the current access source IP/session ID, such as the IDC details of the access source.
IDC details: If the information of the access source includes the IDC, the IDC information will be displayed.
Threat Intelligence: If the IP information of the access source is matched, the matched tag and its definition will be displayed.
AI evaluation
The AI evaluation tab displays the following feature metrics detected exceptionally and the corresponding probability values, including information of Cookie, User-Agent, Referer, and Query. If a metric's value larger than 0, this metric is considered exceptional.
Bot flow statistics
The bot flow statistics tab displays the feature metrics detected exceptionally, and the corresponding values estimated, as well as the reference values.
Field description:
Metric
Description
Session average speed
This metric indicates that the session average speed is considered exceptional, and gives a probability threshold to confirm the exception.
User-Agent type
This metric indicates that the User-Agent type is considered exceptional, and gives a probability threshold to confirm the exception.
URL type
This metric indicates that the URL type is considered exceptional, and gives a probability threshold to confirm the exception.
Session duration
This metric indicates that the session duration is considered exceptional, and gives a probability threshold to confirm the exception.
Total session count
This metric indicates that the total session count is considered exceptional, and gives a probability threshold to confirm the exception.
Session management
The session management tab displays the IP addresses accessed by the current session, the number of accesses by each IP address, and the access logs of the current session ID.
Note:
When session settings are configured, the session management tab will be displayed.
Yes
No
Was this page helpful?