Overview
Cloud Log Service (CLS) provides a one-stop log data solution. You can quickly and conveniently connect to it in five minutes to enjoy a full range of stable and reliable services from log collection, storage, and processing to search, analysis, consumption, shipping, dashboard generation, and alarming, with no need to care about resource issues such as scaling. It helps you improve the problem locating and metric monitoring efficiency in an all-around manner, making log Ops much easier.
This document describes how to use basic CLS features:
Collect log files from servers with LogListener
Search for and analyze logs
If you don't have proper resources to collect logs, you can use demos to quickly try out CLS's log search and analysis, dashboard, and alarming features free of charge without collecting logs. Step 1. Activate the service
Log in to the Tencent Cloud CLS console. If CLS is not activated for your account, you will be prompted for activation. Just click Activate. Step 2. Install LogListener
Step 2.1. Get the key
Go to the CAM console, view/create and record the key, and make sure that the key is enabled. Step 2.2. Install LogListener
1. On the Machine Group Management page, switch to the target CVM/LightHouse region in the top-left corner and click Deploy Instances in the top-right corner. 2. Select the target instance, enter the SecretId and SecretKey obtained in step 2.1 in Enter a SecretId, and enter Machine label (such as test, which is similar to an instance category for subsequent batch log collection from multiple machines).
3. After the installation is completed, click Next.
4. Add the instance with LogListener installed to a new machine group that requires log collection. Log files under the same path can be batch collected for instances in the same group. Enter the Machine Group Name and click Join.
Step 3. Create a log topic
A log topic is the basic unit for log data collection, storage, search, and analysis. It usually corresponds to a certain application/service (with a similar log structure). Log topics can be grouped by logset. A logset doesn't store any log data and is only used to facilitate log topic management.
1. On the Log Topic page, switch to the region in step 2.2 in the top-left corner and click Create Log Topic. 2. In the pop-up window, enter information and click OK.
Log Topic Name: test for example
Storage Class: STANDARD
Logset Operation: Create Logset
Logset Name: test for example
1. On the Log Topic page, click the Log Topic Name/ID in step 3. 2. Select the Collection Configuration tab and click Add in the LogListener Collection Configuration area.
3. On the Log Data Source page, select ** Logs with Full Text in a Single Line**.
Note:
If you select Logs with Full Text in a Single Line, raw logs will be directly reported to CLS, and log fields won't be segmented or extracted. This is a simple way of extraction suitable for getting started with CLS, but it may prevent you from using features such as log search and analysis (for example, log search by field or statistical log analysis). In actual use, we recommend you select a proper log format to segment and extract log fields as instructed in Collection Overview. For JSON logs, you can select JSON Log File.
4. Select the machine group created in step 2.2 and click Next.
5. Enter the Collection Rule Name and Collection Path (i.e., the path of the target log file) and click Next.
For example, if the absolute path of the target file is /root/test.log, then the Directory Prefix for Collection Path should be /root, and the file name should be test.log.
6. Set the index configuration and enable full-text index.
Note:
If Extraction Mode is not Full text in a single line, you can enable Key-Value Index and click Auto Configure to automatically configure the key-value index for the collected logs.
The modified index configuration takes effect only for newly written logs. Existing data won't be updated.
Step 5. Search for and analyze logs
1. On the Search and Analysis page, select the log topic created in step 3 at the top to view the collected log data. 2. In the input box at the top, enter the target text as the search condition and click Search and Analysis to search for logs matching the condition.
3. Use the pipe symbol and SQL for statistical analysis of the found raw data.
For example, calculate the distribution of log sources.
Note:
__SOURCE__ is a system preset field indicating the source IP of a log. After structuring a log and enabling statistics in Key-Value Index for log fields, you can perform statistical analysis on log fields, such as log count by URL.Concepts: This document describes the basic concepts of CLS, including log topic, logset, index, and segment. Creating Processing Task: The data processing feature provides the capabilities to filter, cleanse, mask, enrich, and distribute raw logs. Monitoring Alarm Overview: An alarm policy can be set for logs, for example, triggering an alarm when the number of error logs exceeds 10 within one minute.