The access log feature is used to record access log information for domain names protected by SaaS WAF. It provides access log recording, querying, and downloading capabilities for domains with the switch for access logs enabled within user-defined retention periods (7 to 184 days). After enabling this feature, you can query and download access logs as needed.
Note:
If you need to use the access log feature, please first purchase Log storage packages and follow the operation steps to enable the switch for access logs. Please note that WAF only records access logs for domains with the switch for access logs enabled.
If you need to disable the access log feature, please find the corresponding domain name on the Domain Onboarding page and disable the access log feature.
If you need to disable auto-renewal for the access log feature, please find the corresponding billing item for the service package for security logs on the Renewal Management page and cancel its auto-renewal.
After the service package expires, the system will stop storing new access logs.
After the resources are destroyed, all historical logs will be cleared within 24 hours. This operation is irreversible, so please proceed with caution.
After the service package for security logs expires, the resources will be retained for 7 days. Renewals made during this period are considered as renewals, with the billing cycle starting from the original expiration date. If no renewal is made after this period, the log resources will be destroyed, and subsequent purchases will be considered as new orders.
When the stored log volume exceeds the purchased capacity, the system will automatically stop collecting new access logs. Historical logs will still be retained until they are automatically deleted upon reaching the preset storage period. To prevent new logs from being lost due to exceeded log volume, we recommend that you regularly monitor your usage of the log storage capacity and expand your storage capacity in a timely manner to ensure complete recording of access logs.
Operation Steps
Enable Access Log
Log in to the WAF console. In the left sidebar, choose Connection Management > Domain Onboarding to go to the Connection Management page. Then, select a domain name in the domain list and click
to enable the Access Logs switch.
Configure Storage for Access Logs
Note:
The full configuration for log storage is only displayed when "All instances" and "All domains" are selected. When "Single domain" is selected, you can only modify the settings for the log storage fields for that domain.
1. Log in to the WAF console, select Access Logs in the left sidebar, and click Log collection.
2. On the Log collection page, you can switch between instances and domains in the upper-left corner. Click Configuration storage in the upper-right corner to view and modify configurations for log storage.
Valid Domain Scope: allows you to view the number of domains for which access logs are enabled. You can click Set now to enable or disable the access log switch for individual domains in the domain list.
Log Retention Period: Click Edit to modify the log retention period. You can set the desired retention period within the range of 7 to 184 days. The storage duration can be modified once every two months.
Note:
After the log storage duration is modified, logs stored before the modification will be purged according to the original storage duration, while newly stored logs will be purged based on the modified storage duration.
Log Storage Field: Click Edit to select whether to save BOT information, request content Request Body, and custom Headers.
Note:
Settings for log fields can be configured for all domains or individual domains. When policies are configured for both all domains and individual domains, the policy configured for the individual domain takes precedence.
Log Clearing Quota: Click Clear to delete all currently stored historical logs. Some statistics and report data will be discarded, and this operation cannot be undone. A maximum of 4 manual clearings are allowed per calendar month.
Note:
Manual clearing applies only to all currently stored logs. This operation takes approximately 10 minutes, and log ingestion will be suspended during this operation.
Storage Alarm Settings: Click Edit to set the notification threshold percentage. When the log storage reaches the threshold percentage you set, alarm notifications will be triggered via SMS, in-site messages, emails, WeChat, and other channels for the current account.
Note:
Alarm frequency: After the set percentage is reached, alarm messages for log storage will be sent a maximum of 1 time per day to avoid excessive notifications.
Receiving channels and recipient settings: To modify message recipients or receiving methods, please go to Recipient Management to configure.
3. On the Log collection page, you can view the log usage progress bar in the top-right corner. Click Learn more to jump to the WAF billing details page.
Search Access Logs
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection to switch to the Log collection page.
3. Before searching access logs, please first set the search scope. Select the instance and domain, set the time range, and click
.
Interactive mode: Access Logs are searched based on interaction criteria.
3.1.1 On the Access logs > Log collection page, select Interactive mode.
3.1.2 Click Add Search Condition, select fields of log details and logical relations, then click OK. For descriptions of log details fields, see Field Descriptions for Log Details.
3.1.3 Repeat the previous step until all search conditions are added, then click
.
Statement mode: Search access logs based on search statements.
3.1.4 On the access logs > Log collection page, choose statement mode.
3.1.5 You can write query statements in the following two ways:
Enter search statements directly in the statement box, and then click
to query. For details about search syntax, see Syntax Rules.
Click AI Intelligent Writing beside the input box for query statements. Enter your query requirements, click Send or press the Enter key, and the system will generate the query statement for you.
Analyze Access Logs
Raw Log
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection > Raw logs.
Above the raw logs, you can view key information such as the total number of logs matching the current search conditions and their time distribution. You can also change the display style of raw logs using the settings panel below.
On the left of the raw log data list, click "field name" to display the TOP 5 matching field details sorted by number of logs, along with their percentage of logs. For log details field descriptions, see Field Descriptions for Log Details.
In the access logs data list, click
to the left of the occurrence time of each displayed log to view field details; click JSON to view field details in JSON format. For log details field descriptions, see Field Descriptions for Log Details.
Chart
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection > Chart.
3. When generating charts, you can select:
Interactive mode: Generate charts by adding statistical statements. Click Add Statistical Statement, configure specific metrics, dimensions, sorting methods, and statistical approaches; set the maximum number of returned results, and click Confirm.
Statement Mode: Directly use query statements to generate charts:
Enter search statements directly in the statement box, and then click
to query. For details about search syntax, see Syntax Rules.
Click AI Intelligent Writing beside the input box for query statements. Enter your query requirements, click Send or press the Enter key, and the system will generate the query statement for you.
4. After the chart is generated, you can adjust its presentation through the following two methods:
In chart configuration, directly modify the chart type.
Use the styles or statement templates provided by chart recommendations to quickly optimize the presentation.
Download Access Logs
1. Log in to WAF console, select Access Logs in the left sidebar.
2. On the access logs page, click Log collection> Raw logs.
3. In the upper-right corner of the raw logs data list, click
to slide out the download tasks page.
Click Download Logs to go to the Download Log Data page. Configure data format, log sorting, selected fields, log quantity, and other options as needed, then click Export.
Note:
The scope of the current search logs is downloaded by default.
Only one download task can be created within the same time period. Please wait patiently.
A maximum of 1 million logs can be downloaded in a single task. If you need to download more than 1 million logs, it is recommended to split the download into multiple tasks.
When a wildcard domain (such as: *.abc.com) is selected, logs from all associated subdomains (ending with .abc.com) will also be downloaded.
A maximum of five download tasks can be created. Please note the number of download tasks.
Click Download Records to go to the download records page. Here, you can view all information related to download tasks and perform delete or download operations on completed download tasks.
Note
Successfully created tasks for downloading logs are retained for 3 days. Log files will be deleted after 3 days. Please download them in a timely manner.
Log shipping
Log delivery supports all field data from access logs currently collected by the WAF engine. All you need is simple configuration in the WAF console to complete the near real-time delivery service for access log data. For details about log delivery, see Log shipping.
Appendix
Field Descriptions for Log Details
Information Type
Field Name
Description
Basic Information
domain
Domain: The wildcard domain it belongs to.
request_time
Request time: The time required from the client request arriving at WAF to the response returning from WAF.
client
IP address of the access source: the source IP address of client requests.
uuid
Request UUID: the unique identifier for HTTP requests.
schema
Request protocol: HTTP or HTTPS.
method
Method for client requests.
instance
WAF instance name associated with the domain.
edition
Types of WAF instances for domain access: include sparta-waf (SaaS-based WAF) and clb-waf (CLB-based WAF).
sec_chain
Security modules through which the request passed and the corresponding actions taken.
appid
APPID of the user's Tencent Cloud account.
Information about the access IP address.
ipinfo_nation
Country of origin of the IP address used for access.
ipinfo_state
Country of origin of the IP address used for access, in ISO country code format.
ipinfo_city
City of origin for the IP address used for access.
ipinfo_province
Province of origin for the IP address used for access.
ipinfo_isp
ISP for the IP address used for access.
ipinfo_detail
Details of the IP address used for access.
ipinfo_longitude
Longitude information associated with the IP address used for access.
ipinfo_dimensionality
Latitude information associated with the IP address used for access.
Request details.
url
Request uri: refers to the content between the first "/" after the domain name and the "?" in the client's complete request path.
accept
Accept: HTTP request header field used to inform the server of the response content types supported by the client.
encoding
Accept-Encoding: HTTP request header field used to inform the server of the compression algorithms supported by the client.
language
Accept-Language: HTTP request header field used to inform the server of the languages supported by the client.
connection
Connection: HTTP request header field used to control connection behaviors, such as keep-alive, closing connections, and so on.
content_type
Content-Type: HTTP request header field that specifies the MIME type of the request body.
cookie
Request cookie: information with a maximum length of 1K.
host
Domain requested by the client.
referer
Source page.
x_forwarded_for
X-Forwarded-For: records all proxy IP addresses traversed by the client request and the client's real IP address.
user_agent
Request UA: indicates the software and operating system information of the client.
headers
Information of the HTTP request header.
request_length
Request size, upstream bandwidth.
query
Query String: HTTP request parameter with a maximum length of 1K.
msec
Timestamp: when the request occurs.
time
NGINX local time string.
Response details.
upstream_status
Response status code returned from the origin server to the Web Application Firewall.
status
Response status code returned to the client by SaaS-based WAF:
200: OK
202: Frontend Countermeasure
302: Redirect
403: Block
4XX: Refer to the standard definitions of HTTP response status codes.
5XX: Refer to the standard definitions of HTTP response status codes.
Response status code returned by the Load Balancer-based Web Application Firewall to the Load Balancer:
600: OK
624: Frontend Countermeasure
621: Redirect
615: Block
bytes_sent
Response body size.
upstream_connect_time
Connection time from WAF to origin server.
upstream_response_time
Time required for client requests to return from the origin server to WAF.
upstream
Upstream server IP address.
information about BOT traffic
bot_module
BOT detection module hit by current access request.
bot_action
BOT action taken for the current access request.
bot_score
information about the BOT score of the current access request.
bot_label
BOT Tag hit by current access request.
ua_type
User-Agent type of the accessing user in the current request.
ua_crawlername
Name of the User-Agent suspected to be a crawler in the current access request.
ua_fake
Whether the User-Agent in the current access request is forged, 0 for no, 1 for yes.
ua_goodbot
Whether the BOT in the current access request is goodbot, 0 for no, 1 for yes.
bot_ai
Whether the current access request is flagged as abnormal by the AI engine, 0 for normal, 1 for abnormal.
bot_stat
Whether the current access request is identified as abnormal by intelligent statistics, 0 for normal, 1 for abnormal.
bot_ti_tags
Whether the current access request matches threat intelligence, displaying the matched intelligence Tag.
bot_id
BOT ID of the current access request.
bot_scene_id
ID of the BOT scenario hit by the current access request.
bot_action_id
ID of the BOT action policy hit by the current access request.
bot_rule_id
ID of the BOT rule hit by the current access request.
bot_rule_name
Name of the BOT rule hit by the current access request.
bot_token
ID of the BOT session for the current access request.
