With log shipping, you can check logs of multiple security products in the CSC console, and ship logs to specific Kafka systems for log storage and analytics.
Use cases
Log storage
To ensure data security, it's recommended to store logs for at least six months.
Offline analytics
After the logs are shipped to a Kafka system, you can analyze logs with other tools, which help find out the root of problems and discover vulnerablities.
Prerequisite
Purchase CSC Ultimate and connect the logs of related products to CSC. To ship logs to CKafka, you need to create a CKafka instance first.
1. Log in to the CSC console and select Log audit from the left sidebar.
2. On the Log audit page, click Log shipping.
3. CSC automatically obtains the CKafka instances under the current account and the log sources connected to CSC. Complete the configuration as instructed below.
Parameter
Description
Destination
Select CKafka (public domain).
Kafka instance
CSC automatically obtains the CKafka instances under your account. Select the instance you need.
Public domain
Select the public domain name.
Username
Enter the username used to access the Kafka instance.
Password
Enter the password used to access the Kafka instance.
Delete: Locate the configuration to delete and click Delete under Operation.
Edit: Click Modify configuration on the log shipping page to modify the relevant log shipping configuration.
4. Click OK to ship collected logs to the specified Kafka instance.
5. On the Log shipping page, you can view details of the destination Kafka instance, such as the destination access method, address, status, username, log source, log type, account source (when Multi-account configured), Topic ID/name, shipping status and configuration status. You can modify the destination Kafka instance and log topic configurations.
Connecting to CKafka over a Support Environment
1. Log in to the CSC console and select Log audit from the left sidebar.
2. On the Log audit page, click Log shipping.
3. CSC automatically obtains the CKafka instances under the current account and the log sources connected to CSC. Complete the configuration as instructed below.
Parameter
Description
Destination
Select CKafka (supporting environment).
Kafka instance ID
CSC automatically obtains the CKafka instances under your account. Select the instance you need.
Delete: Locate the configuration to delete and click Delete under Operation.
Edit: Click Modify configuration on the log shipping page to modify the relevant log shipping configuration.
4. Click OK to ship collected logs to the specified Kafka instance.
5. On the Log shipping page, you can view details of the destination Kafka instance, such as the destination access method, address, status, username, log source, log type, account source (when Multi-account configured), Topic ID/name, shipping status and configuration status. You can modify the destination Kafka instance and log topic configurations.
Connecting to an external Kafka system
1. Log in to the CSC console and select Log audit from the left sidebar.
2. On the Log audit page, click Log shipping.
3. On the Log shipping page, select External Kafka (public network) and complete the parameters as instructed below.
Parameter
Description
Destination
Select External Kafka (public network).
Public network
Enter the public network address of the external Kafka.
Username
Enter the username used to access the Kafka instance.
Password
Enter the password used to access the Kafka instance.
Delete: Locate the configuration to delete and click Delete under Operation.
Edit: Click Modify configuration on the log shipping page to modify the relevant log shipping configuration.
4. Click OK to ship collected logs to the specified Kafka instance.
5. On the Log shipping page, you can view details of the destination Kafka instance, such as the destination access method, address, status, username, log source, log type, account source (when Multi-account configured), Topic ID/name, shipping status and configuration status. You can modify the destination Kafka instance and log topic configurations.
Managing log shipping
Multi-account
After Multi-account management is enabled, you can manage log shipping configurations under multiple accounts.
1. Log in to the CSC console and select Log audit from the left sidebar.
2. On the Log audit page, click Multi-account in the upper right corner.
3. Select accounts as needed and click OK.
Scenario
How-to
Outcome
The administrator/delegated administrator needs to ship logs of multiple products under all accounts to the same Kafka destination.
Select all accounts from the Multi-account drop-down list. Select CKafka (public domain) or CKafka (supporting environment) for Destination. CKafka instances under the administrator account are automatically obtained and listed. Select the destination CKafka instance as needed.
Display the information of CKafka instances under the administrator account. Details of log shipping configurations are synched automatically.
The administrator/delegated administrator needs to manage logs of specified accounts.
Select target accounts from the Multi-account drop-down list. Select CKafka (public domain) or CKafka (supporting environment) for Destination. CKafka instances under the target accounts are automatically obtained and listed. Select the destination CKafka instance as needed.
Display the information of CKafka instances under the target accounts. Details of log shipping configurations are synched automatically.
The administrator/delegated administrator needs to manage the logs of the current account (administrator/delegated administrator).
Select the current administrator/delegated administrator account from the Multi-account download list. Select CKafka (public domain) or CKafka (supporting environment) for Destination. CKafka instances under the current administrator/delegated administrator account are automatically obtained and listed. Select the destination CKafka instance as needed.
Display the information of CKafka instances under the current administrator/delegated administrator account. Details of log shipping configurations are synched automatically.
Single account
Ship logs of products under the current account.
How-to: Complete log shipping configurations and select CKafka (public domain) or CKafka (supporting environment) for Destination. CKafka instances under the current account are automatically obtained and listed. Select the destination CKafka instance as needed.
Note:
If the current account is managed by an administrator/delegated administrator, the administrator/delegated administrator can modify the log shipping configuration of the current account.
Outcome: Display the information of CKafka instances under the current account. Details of log shipping configurations are synched automatically.
Which IPs should be allowed for log shipping over the public network?
106.55.200.0/24
106.55.201.0/24
106.55.202.0/24
81.71.5.0/24
134.175.239.0/24
193.112.130.0/24
193.112.164.0/24
193.112.221.0/24
111.230.173.0/24
111.230.181.0/24
129.204.232.0/24
193.112.129.0/24
193.112.153.0/24
106.52.11.0/24
106.55.52.0/24
118.89.20.0/24
193.112.32.0/24
193.112.60.0/24
106.52.106.0/24
106.52.67.0/24
106.55.254.0/24
42.194.128.0/24
42.194.133.0/24
106.52.69.0/24
118.89.64.0/24
129.204.249.0/24
182.254.171.0/24
193.112.170.0/24
106.55.207.0/24
119.28.101.0/24
150.109.12.0/24
Which products and log types are supported for log shipping?
Product
Log type
Description
Cloud Firewall (CFW)
Access control logs
Logs of hits on access control rules for edge firewalls, NAT firewalls, inter-VPC firewalls, and enterprise security groups.
Intrusion prevention logs
Logs of security events detected by the Observe and Block policies. Event types include intrusions, compromised servers, lateral movements, and network honeypots.
Traffic logs
Logs of north-south traffic generated by edge firewalls and NAT firewalls based on outbound and inbound traffic, as well as east-west traffic between VPCs.
Operation logs
Logs of all operations performed on the security policies and toggles
Web Application Firewall (WAF)
Attack logs
Logs of attacks, including the attack time, attacker IP, attack type and other attack details.
Access logs
Logs of access to domain names.
Cloud Workload Protection Platform (CWPP)
Intrusion detection logs
Logs of Trojans, high-risk commands, local privilege escalation and all abnormal login events.
Vulnerability management logs
Security log of vulnerability security events
Advanced defense logs
Logs of advanced features, including Java Webshell and attack detection.
Agent logs
Abnormal CWPP agent events, including the agent being offline for over 24 hours and the agent being uninstalled (only for Linux servers).
Yes
No
Was this page helpful?