Real-time Log Analysis enables quick retrieval, analysis, and storage of log data through the real-time collection and delivery of access logs from Cloud Streaming Services to the Cloud Log Service (CLS) product. By mining log data, it supports data-driven operations and management, allowing for the rapid and accurate development of operational strategies.
Note
Real-time Log Analysis now fully supports shipping logs to Cloud Log Service. You are welcome to experience this. This documentation will guide you on how to use the latest real-time log feature.
Notes
Log data is collected in real-time, with log search and reporting data stabilizing after three minutes.
Currently, reporting analysis is only available for playback logs. If you have other log management needs, visit Cloud Log Service.
After enabling the log delivery feature, ensure that Cloud Log Service is in normal operation, as the suspension of Cloud Log Service will prevent the delivery of logs.
The bandwidth or traffic data recorded in logs is the application layer (HTTP protocol) return data. Due to mechanisms like TCP header consumption and failed retransmissions, it is smaller than the bandwidth or traffic consumption calculated at the TCP layer.
Operation Instruction
Creating a log topic
1. Go to the Cloud Streaming Services console, and select Monitoring > Cloud Log Service > Real-time log analysis, to enter real-time log analysis.
2. For first use, authorization of the Cloud Streaming Services service role is required. Once the role is authorized, you need to agree to the service agreement and click Start. The system will automatically activate the Cloud Log Service (CLS) product and open the real-time log analysis management page.
3. Choose a region and create a log set, click Here on the page to create a new one.
Note:
The region includes Guangzhou and Singapore. Log topics created under the log set in the Guangzhou region can only deliver logs within the Chinese mainland. In contrast, log topics created under the log set in the Singapore region can only deliver logs internationally / to Hong Kong (China), Macao (China), and Taiwan (China).
4. Click Confirm to create a new log set.
5. After successfully creating a log set, click Creating Log Topic to enter the log topic creation page. The newly created log topic will by default be in the process of delivering logs to the Cloud Log Service (CLS) product.
Modifies a log topic
1. Enter the log topic list in real-time log analysis and click Manage in the operation column of the log topic you need to modify.
2. You can enter the log topic editing page to modify the log topic information.
Analyzing a log report
Only log topics of the log type provide report analysis. There are four types of data on the page: Basic Data Analysis, Resource Distribution Analysis, Exception Diagnosis Analysis, and User Analysis.
1. Enter the log topic list in Real-time Log Analysis andclick Report on the right side of the log topic you want to view.
2. Enter the log report page to view report data; you can separately view Basic Data, Resource Distribution, Exception Diagnosis, and User Analysis.
Basic Data
Resource Distribution
Exception Diagnosis
User Analysis
Log Search
Log search supports various types of search analysis methods and chart analysis forms. For detailed information, see Log Search and Analysis.
Perform log search by log topic as a cell. Select the log topic you need to search for and click Search to enter the log search page.
Stop shipping log topic to CLS
1. Enter the log topic list in Real-time Log Analysis, and click Stop on the right of the log Topic you wish to stop shipping.
2. In the pop-up window, confirm whether to stop shipping and click Confirm to proceed. The status of the corresponding log topic will change to stop shipping, and logs will no longer be shipped to the CLS product.
Deleting a log topic
Note:
Once a log topic is deleted, it cannot be recovered. Please proceed with caution.
If you delete a log topic, you will stop pushing log data of the related domain, and the corresponding log topic in the Cloud Log Service product will also be deleted and all shipped logs will be cleared. Meanwhile, you will no longer be able to use the report associated with that topic.
1. Enter the log topic list in Real-time Log Analysis and click Delete on the right side of the log topic you wish to remove.
2. In the pop-up window, confirm whether to delete the log topic and click OK to proceed.
Log Field
Streaming Log
Order
Log Field
Description
1
time
Request time
2
client_ip
Client IP
3
host
Accessed domain name
4
url
URL
5
size
Streaming bytes size
6
country_id
country_id
7
prov
Province
8
isp
ISP
9
streamname
Stream ID
10
node_ip
Node IP
11
server_region
Server region
12
server_country
Server country
Playback Log
Order
Log Field
Description
1
type
Playback type: lvb represents Live Video Broadcasting and leb represents Live Event Broadcasting
2
time
Request time
3
client_ip
Client IP
4
host
Accessed domain name
5
url
URL
6
size
Number of bytes of this access request
7
country_id
country_id
8
prov
Province
9
isp
ISP
10
http_code
HTTP status code
11
referer
Referer information
12
process_time
Processing duration (in milliseconds)
13
ua
User-Agent information
14
range
Range parameter
15
method
HTTP Method
16
streamname
Stream ID
17
hit
Cache hit/miss
18
node_ip
Node IP
19
server_region
Server region
20
server_country
Server country
21
connect_fd
connect_fd (connection port number)
22
lost_rate
The packet loss rate, only valid for type=leb
23
rtt
rtt, only valid for type=leb
Definitions
Logset
A log set (Logset) classifies log topics and metric topics; a log set can contain multiple log topics and metric topics. A log set itself does not store any log data, it only facilitates user management of topics. Cloud Streaming Services log sets have the following basic attribute information:
Guangzhou and Singapore regions are currently supported.
Logset Name: The name of a logset.
Retention period: The default retention period for data in the log set is 30 days.
Creation time: Logset creation time.
Log Topic
A log topic (Log Topic) is a basic unit for log data collection, storage, retrieval, and analysis on the Cloud Log Service (CLS) platform. The vast amounts of logs collected are managed by log topic, including the configuration of the collection rules and storage time, log search and analysis, and log download/consumption/delivery.
Log topic features include:
Collect logs to log topics.
Store and manage logs based on log topics.
Search and analyze logs by log topics.
Ship logs to other platforms based on log topics.
Download and consume logs from log topics.
Note
The information above is excerpted from the Cloud Log Service (CLS) product documentation. For more details, see Log Topic and Logset and refer to the Cloud Log Service (CLS) documentation for accurate information.
Yes
No
Was this page helpful?