The access log feature is used to record access log information for domain names protected by SaaS WAF. It provides access log recording, querying, and downloading capabilities for domains with the switch for access logs enabled within user-defined retention periods (7 to 184 days). After enabling this feature, you can query and download access logs as needed.
Note:
If you need to use the access log feature, please first purchase Log storage packages and follow the operation steps to enable the switch for access logs. Please note that WAF only records access logs for domains with the switch for access logs enabled.
If you need to disable the access log feature, please find the corresponding domain name on the Domain Onboarding page and disable the access log feature.
If you need to disable auto-renewal for the access log feature, please find the corresponding billing item for the service package for security logs on the Renewal Management page and cancel its auto-renewal.
After the service package expires, the system will stop storing new access logs.
After resource destruction, all historical logs will be cleared within 24 hours. This operation is irreversible, please proceed with caution.
After the service package for security logs expires, the resources will be retained for 7 days. Renewals made during this period are considered as renewals, with the billing cycle starting from the original expiration date. If no renewal is made after this period, the log resources will be destroyed, and subsequent purchases will be considered as new orders.
When the stored log volume exceeds the purchased capacity, the system will automatically stop collecting new access logs. Historical logs will still be retained until they are automatically deleted upon reaching the preset storage period. To prevent new logs from being lost due to exceeded log volume, we recommend that you regularly monitor your usage of the log storage capacity and expand your storage capacity in a timely manner to ensure complete recording of access logs.
Operation Steps
Enable Access Log
Log in to the WAF console. In the left sidebar, choose Connection Management > Domain Onboarding to go to the Connection Management page. Then, select a domain name in the domain list and click
to enable the Access Logs switch.
Configure Storage for Access Logs
Note:
The full configuration for log storage is only displayed when "All instances" and "All domains" are selected. When "Single domain" is selected, you can only modify the settings for the log storage fields for that domain.
1. Log in to the WAF console, select Access Logs in the left sidebar, and click Log collection.
2. On the Log collection page, you can switch between instances and domains in the upper-left corner. Click Configuration storage in the upper-right corner to view and modify configurations for log storage.
Valid Domain Scope: allows you to view the number of domains for which access logs are enabled. You can click Set now to enable or disable the access log switch for individual domains in the domain list.
Log retention period: Click Edit to modify the log retention period. Set the desired duration between 7 to 184 days. The storage duration can be modified once every two months.
Note:
After the log storage duration is modified, the system will process each log entry according to its respective retention policy:
Logs stored before the modification will be automatically deleted upon expiration of the original storage duration.
Newly generated logs after the modification will be automatically deleted upon expiration of the new storage duration.
Log Storage Field: Click Edit to select whether to save BOT information, request content Request Body, and custom Headers.
Note:
Settings for log fields can be configured for all domains or individual domains. When policies are configured for both all domains and individual domains, the policy configured for the individual domain takes precedence.
Number of log clearances: Click Manual Clearance to delete all currently stored historical logs. Some statistics and report data will be discarded, and this operation is irreversible. A maximum of 4 clearance operations are allowed per calendar month.
Note:
Manual clearance applies only to all current logs and the operation is expected to take approximately 10 minutes, during which log ingestion will be suspended.
Storage Alarm Settings: Click Edit to set the notification threshold percentage. When the log storage reaches the threshold percentage you set, alarm notifications will be triggered via SMS, in-site messages, emails, WeChat, and other channels for the current account.
Note:
Alarm frequency: After the set percentage is reached, alarm messages for log storage will be sent a maximum of 1 time per day to avoid excessive notifications.
Receiving channels and recipient settings: To modify message recipients or receiving methods, please go to Recipient Management to configure.
3. On the Log collection page, you can view the log usage progress bar in the top-right corner. Click Learn more to jump to the WAF billing details page.
Search Access Logs
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection to switch to the Log collection page.
3. Before searching access logs, please first set the search scope. Select the instance and domain, set the time range, and click
.
Interactive mode: Access Logs are searched based on interaction criteria.
3.1.1 On the Access logs > Log collection page, select Interactive mode.
3.1.2 Click Add Search Condition, select fields of log details and logical relations, then click OK. For descriptions of log details fields, see Field Descriptions for Log Details.
3.1.3 Repeat the previous step until all search conditions are added, then click
.
Statement mode: Search access logs based on search statements.
3.1.4 On the access logs > Log collection page, choose statement mode.
3.1.5 You can write query statements in the following two ways:
Enter search statements directly in the statement box, and then click
to query. For details about search syntax, see Syntax Rules.
Click AI Intelligent Writing beside the input box for query statements. Enter your query requirements, click Send or press the Enter key, and the system will generate the query statement for you.
Analyze Access Logs
Raw Log
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection > Raw logs.
Above the raw logs, you can view key information such as the total number of logs matching the current search conditions and their time distribution. You can also change the display style of raw logs using the settings panel below.
On the left of the raw log data list, click "field name" to display the TOP 5 matching field details sorted by number of logs, along with their percentage of logs. For log details field descriptions, see Field Descriptions for Log Details.
In the access logs data list, click
to the left of the occurrence time of each displayed log to view field details; click JSON to view field details in JSON format. For log details field descriptions, see Field Descriptions for Log Details.
Chart
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection > Chart.
3. When generating charts, you can select:
Interactive mode: Generate charts by adding statistical statements. Click Add Statistical Statement, configure specific metrics, dimensions, sorting methods, and statistical approaches; set the maximum number of returned results, and click Confirm.
Statement Mode: Directly use query statements to generate charts:
Enter search statements directly in the statement box, and then click
to query. For details about search syntax, see Syntax Rules.
Click AI Intelligent Writing beside the input box for query statements. Enter your query requirements, click Send or press the Enter key, and the system will generate the query statement for you.
4. After the chart is generated, you can adjust its presentation through the following two methods:
In chart configuration, directly modify the chart type.
Use the styles or statement templates provided by chart recommendations to quickly optimize the presentation.
Download Access Logs
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection> Raw logs.
3. In the upper-right corner of the raw logs data list, click
to slide out the download tasks page.
Click Download Logs to go to the Download Log Data page. Configure data format, log sorting, selected fields, log quantity, and other options as needed, then click Export.
Note:
The scope of the current search logs is downloaded by default.
Only one download task can be created within the same time period. Please wait patiently.
A maximum of 1 million logs can be downloaded in a single task. If you need to download more than 1 million logs, it is recommended to split the download into multiple tasks.
When a wildcard domain (such as: *.abc.com) is selected, logs from all associated subdomains (ending with .abc.com) will also be downloaded.
A maximum of five download tasks can be created. Please note the number of download tasks.
Click Download Records to go to the download records page. Here, you can view all information related to download tasks and perform delete or download operations on completed download tasks.
Note
Successfully created tasks for downloading logs are retained for 3 days. Log files will be deleted after 3 days. Please download them in a timely manner.
Log shipping
Log delivery supports all field data from access logs currently collected by the WAF engine. All you need is simple configuration in the WAF console to complete the near real-time delivery service for access log data. For details about log delivery, see Log shipping.
Appendix
Field Descriptions for Log Details
Information Type
Field Name
Description
Basic Information
domain
Domain name accessed by client requests. In cases of wildcard domain names or object-based access, the current domain name is the exact one.
request_time
Request latency: the time taken for a client request to reach the Web Application Firewall (WAF) and return from the WAF. Unit: seconds.
client
IP address of the access source: the source IP address of client requests.
uuid
Request UUID: the unique identifier for HTTP requests.
schema
Request protocol: HTTP or HTTPS.
method
Method for client requests.
instance
ID of the Web Application Firewall instance to which it belongs.
query
Query String of the client HTTP request, maximum length is 1K Byte.
time
The time when the client HTTP request was recorded by NGINX, expressed in a locally readable time format.
timestamp
Timestamp in ISO 8601 format for when the client HTTP request occurred.
appid
APPID of the user's Tencent Cloud account.
Header details
url
Client HTTP request header field, which records the content between the first "/" after the domain name and "?" in the complete request path.
accept
Client HTTP request header field, used to inform the server of the response content types supported by the client.
encoding
Client HTTP request header field, used to inform the server of the compression algorithms supported by the client.
language
Client HTTP request header field, used to inform the server of the languages supported by the client.
connection
Client HTTP request header field that controls connection behavior, such as keep-alive, disable connection, and so on.
content_type
Client HTTP request header field that specifies the MIME type of the request body.
cookie
Client HTTP request header field that records the Cookie information of the request, maximum length is 1K. Unit: Byte.
host
Client HTTP request header field that records the requested domain name.
referer
Client HTTP request header field that records the source URL information of the request. If the request has no source URL information, this field displays -.
x_forwarded_for
Client HTTP request header field that records all proxy IP addresses passed through by the client request and the client's real IP address.
user_agent
Client HTTP request header field that records the client's software and operating system information.
request_length
Size of the client HTTP request. Unit: Byte.
Response Details
upstream_status
Response status code returned from the origin server to the Web Application Firewall.
status
Response status code returned to the client by SaaS-based WAF:
200: OK
202: Frontend Countermeasure
302: Redirect
403: Block
4XX: Refer to the standard definitions of HTTP response status codes.
5XX: Refer to the standard definitions of HTTP response status codes.
Response status code returned by the Load Balancer-based Web Application Firewall to the Load Balancer:
600: OK
624: Frontend Countermeasure
621: Redirect
615: Block
bytes_sent
Response body size. Unit: Byte.
upstream_connect_time
Connection time from Web Application Firewall (WAF) to the origin server for client requests. Unit: second.
upstream_response_time
Time taken for the client request to return from the origin server to the Web Application Firewall (WAF). Unit: s.
upstream
IP address of the origin server.
Basic Attack Logs
attack_type
Attack type: the specific attack type detected.
sec_action
Action taken: the mitigation action triggered by client attacks, including four types of handling results: Observe (0), Block (1), CAPTCHA verification (2), Redirect (3).
rule_id
Rule ID: Rule ID that triggered the protection policy.
risk_level
Risk level: risk level triggered by client attacks, including three risk levels: High-risk (1), Medium-risk (2), Low-risk (3).
sec_chain
Security modules through which the request passed and the corresponding actions taken.
BOT Protection Details
bot_module
BOT detection module hit by current access request.
bot_action
BOT action taken for the current access request.
bot_score
information about the BOT score of the current access request.
bot_label
BOT Tag hit by current access request.
ua_type
User-Agent type of the accessing user in the current request.
ua_crawlername
Name of the User-Agent suspected to be a crawler in the current access request.
ua_fake
Whether the User-Agent in the current access request is forged, 0 for no, 1 for yes.
ua_goodbot
Whether the BOT in the current access request is goodbot, 0 for no, 1 for yes.
bot_ai
Whether the current access request is flagged as abnormal by the AI engine, 0 for normal, 1 for abnormal.
bot_stat
Whether the current access request is identified as abnormal by intelligent statistics, 0 for normal, 1 for abnormal.
bot_ti_tags
Whether the current access request matches threat intelligence, displaying the matched intelligence Tag.
bot_id
BOT ID of the current access request.
bot_scene_id
ID of the BOT scenario hit by the current access request.
bot_action_id
ID of the BOT action policy hit by the current access request.
bot_rule_id
ID of the BOT rule hit by the current access request.
bot_rule_name
Name of the BOT rule hit by the current access request.
bot_token
ID of the BOT session for the current access request.
bot_tld_risk_tag
Endpoint risk Tag status for the current access request (requires purchase of RCE TDS capability).
bot_ua
Whether the current access request hit the ua policy.
Access IP address information
ipinfo_nation
Country of origin of the IP address used for access.
ipinfo_state
Country of origin of the IP address used for access, in ISO country code format.
ipinfo_city
City of origin for the IP address used for access.
ipinfo_province
Province of origin for the IP address used for access.
ipinfo_isp
ISP for the IP address used for access.
ipinfo_detail
Details of the IP address used for access.
ipinfo_longitude
Longitude information associated with the IP address used for access.
ipinfo_dimensionality
Latitude information associated with the IP address used for access.
Other custom fields
headers
Protocol header information: including custom header information.
body
Request content: Request Body.
attack_category
Primary attack classification.
attack_content
Attack content: the content that triggered the attack from the client.
attack_place
Attack location: the position of the attack method within the HTTP request.
count
Number of attacks: the number of attacks aggregated every 10 seconds from the same attacker IP address and attack type.
waf_verify
Captcha verification token.
pan
Entry domain or clb object.
http_log
Log file for HTTP request and response information.
args_name
Attack log parameter name: Parameter name in HTTP requests.
