Log Shipping
Last updated:2025-12-19 16:11:26
Ship to CLS
Through the feature for log shipping, you can automatically deliver CFW logs to specified CLS instances to achieve efficient storage and analysis of CFW logs. The following details how to use the feature for log shipping in Log Analysis to CLS.
Note:
To use shipping to CLS, you need to activate Tencent Cloud CLS first, learn about the usage and billing of CLS.
Background
The feature for log shipping can precisely deliver different CFW log types to specified CLS log topics (Log Topic) respectively, facilitating your targeted management and analysis based on log categories.
The feature for log shipping supports two network access methods, namely: public network access and Tencent Cloud VPC private network access.
Leveraging a public network to deliver CFW logs to CLS is applicable to scenarios where direct access to the Tencent Cloud private network is unavailable.
Shipping logs through Tencent Cloud's internal VPC offers advantages in security and transmission performance, effectively ensuring the stability and efficiency of data transmission.
Prerequisites
You need to have activated Tencent Cloud CLS service and CFW Log Analysis. Currently supported log types for shipping are as follows:
Log Type | Log Topic |
internetFlowLog | Traffic logs - Internet boundary |
natFlowLog | Traffic logs - NAT boundary |
vpcFlowLog | Traffic logs - VPC boundary |
dnsFlowLog | Traffic logs - DNS |
eventLog | Intrusion Defense logs |
internetRuleLog | Logs for Access Control - Internet boundary |
natRuleLog | Logs for Access Control - NAT boundary |
vpcRuleLog | Logs for Access Control - VPC boundary |
dnsRuleLog | Logs for Access Control - DNS |
operateLog | Operations Logs |
ndrFlowLog | NDR logs |
Configuration Steps
1. Log in to the Cloud Access Management console by using the root account.
2. In the left sidebar, click User > User List.
3. On the User List page, click Create User.
4. Create a dedicated API account for the shipping tasks of firewall logs:
Note:
It is recommended to use a newly created sub-account for log shipping.
You can use the primary account or a created sub-account to configure CFW log shipping. If using a sub-account, ensure it has the
QcloudCLSFullAccess permission. If not, go to the User > User List page to grant the sub-account QcloudCLSFullAccess.4.1 Customize a username, select Console Login as the access method, and grant full read-write access to CLS:
QcloudCLSFullAccess.4.2 Select users requiring password reset as needed, select tags, and click Creates Users.
5. Log in to the CFW console, and in the left sidebar, click Log Analysis.
6. On the Log Analysis page, click Log Shipping > Ship to CLS.
7. On the Ship to CLS page, click Edit for configuration for CLS shipping, fill in the SecurityID and SecurityKey for authentication, click Save to complete the CLS shipping configuration.
Note:
Security ID can be obtained in Cloud Access Management - API Key Management, and the Security Key is obtained when the account is created.
8. On the Ship to CLS page, locate the Ship option and switch the toggle button from Off to On.
After the above steps are completed, CFW logs will be successfully delivered to CLS. You can further manage and analyze the received log data in the CLS console to better monitor the status of network security and perform related Ops operations.
Ship to Ckafka
Through the log shipping feature, you can automatically deliver CFW logs to specified Ckafka instances. The following details how to use the log shipping feature in Log Analysis.
Background
The log shipping feature can deliver different log types of CFW to specified Ckafka topics respectively.
The log shipping feature supports two network access methods, namely: access to public network domains and access to support environments.
Log shipping via access to public network domains.
Access to the support environment is performed via the Tencent Cloud private network, offering higher performance.
Prerequisites
You need to have purchased Tencent Cloud Message Queue Ckafka instance and CFW Log Analysis. The Ckafka instance's bandwidth specifications should be configured based on the CFW bandwidth.
According to the TDMQ for CKafka guide, contact Tencent Cloud Customer Service to enable the "access to public network domains" or "access to support environments" allowlist.
Only one Ckafka message queue account is supported for log shipping.
Configuration Steps
1. Log in to the CFW console, and in the left sidebar, click Log Analysis.
2. On the Log Analysis page, click Log Shipping to go to the Ship to Kafka page by default.
3. On the Ship to Kafka page, perform the initial configuration.
3.1 The network access methods include: access to public network domains and access to support environments.
Method 1: Select access to domains of public network, choose a message queue instance and domain of public network, and enter the username and password for the selected message queue instance.
Method 2: Select access to support environments, which refers to products you have purchased on Tencent Cloud that can be used in conjunction with Ckafka. Then select a message queue instance and IP address port.
3.2 After the method for network access is selected, you can bind to the Ckafka topic on the log shipping page.
Note:
The log shipping feature supports the shipping of multiple CFW log types. Logs of different types must be delivered to different Ckafka topics. Each Ckafka topic can only be bound to one CFW log type.
3.3 After the configuration is completed, click OK, and a prompt will indicate that configuring log shipping is successful, meaning the log shipping has been set up successfully.
4. Once the initial configuration is completed, you can view the details of log shipping.
Basic Information: Displays the basic information of the Ckafka instance.
Note:
You need to pay attention to the "Health Status" field. When it shows "Unhealthy", click View Monitoring to check whether the Ckafka service is abnormal or whether there is insufficient quota.
Log Ship Switch: Used to control specified log types to start or stop tasks for log shipping.
Method 1: On the right side of each log type in the "Ship Switch" column, you can individually control tasks for log shippingusing the "Switch" button.
Method 2: Use batch operations, currently supporting Start All and Stop All two operations.
Rebind Ckafka topic: In the operation column on the right side of the log type, click Edit to configure individually, allowing you to reselect a Ckafka topic in the specified Ckafka instance that is not bound to other types of firewall logs.
Note:
Each Ckafka topic can only be bound to one log type of CFW.
View Monitoring: In the operation column on the right side of the log type, click View Monitoring to be redirected to the monitoring page of the Message Queue Ckafka console, where you can view network traffic, peak bandwidth, number of messages, disk usage, and other metrics.
Reconfiguration: Above the log type list, click Reconfigure to reselect the message queue instance for shipping, method for network access, and username/password.
Note:
Reconfiguration will interrupt the current shipping process.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No
Feedback