tencent cloud

Cloud Log Service

ドキュメントCloud Log Service

Metric Pre-aggregation

Download
フォーカスモード
フォントサイズ
最終更新日: 2026-05-15 22:05:57
When large volumes of metric data are monitored or complex queries are executed, real-time queries can become slow, and writing query statements may become challenging. To address this, the metric storage system supports a pre-aggregation feature. Pre-aggregation allows for the pre-computation of complex queries (using PromQL) and stores the results as new metrics. These precomputed data can be directly used in subsequent queries and alarms, reducing real-time computation costs, improving query performance, and simplifying and streamlining monitoring configuration. Pre-aggregation is compatible with Prometheus Recording Rules, allowing native YAML configuration files to be directly imported.
For example, the original metric prometheus_http_requests_total records the number of requests made to various Prometheus APIs with different response status codes.
# HELP prometheus_http_requests_total Counter of HTTP requests.
# TYPE prometheus_http_requests_total counter
prometheus_http_requests_total{code="200",handler="/api/v1/label/:name/values"} 7
prometheus_http_requests_total{code="200",handler="/api/v1/query"} 19
prometheus_http_requests_total{code="200",handler="/api/v1/query_range"} 27
prometheus_http_requests_total{code="200",handler="/graph"} 11
prometheus_http_requests_total{code="200",handler="/metrics"} 8929
prometheus_http_requests_total{code="200",handler="/static/*filepath"} 52
prometheus_http_requests_total{code="302",handler="/"} 1
prometheus_http_requests_total{code="400",handler="/api/v1/query_range"} 6
If you need to regularly query the total number of requests for each status code, you can use the following configuration to create a pre-aggregation task:
groups:
  - name: example
    rules:
    - record: code:prometheus_http_requests_total:sum
      expr: sum by (code) (prometheus_http_requests_total)
Among them:
sum by (code) (prometheus_http_requests_total) is the metric query statement (PromQL), which calculates the sum of request counts grouped by status code.
code:prometheus_http_requests_total:sum is the generated metric name, which can be customized. You can use this name in subsequent queries to directly retrieve the precomputed metric.

Application Scenario

Improve metric query performance: When some complex queries need to be executed frequently, pre-aggregation can be used to precompute and store these queries as new metrics. This can significantly enhance query performance and reduce query response time.
Simplify query statements: Complex query statements can be lengthy and difficult to read. By using pre-aggregation, you can store the results of complex queries as simplified metrics. This makes subsequent queries easier to write and maintain.
Store core metrics separately: All data in the metric topic is retained for the same duration and is automatically cleared after expiration. If you need to store a subset of core metrics for a longer period, you can use a pre-aggregation task to store this portion of data in another metric topic.

Prerequisites

A metric topic has already been created.

Directions

Creating Task

1. Log in to the Cloud Log Service console.
2. In the left sidebar, click Metric Topic.
3. Click the metric topic ID/name for which you want to create a pre-aggregation task to enter the metric topic management page.
4. Click the Metric Pre-aggregation tab to enter the pre-aggregation task list page, then click Creating Task. The main configuration items are described as follows:
Configuration Item
Description
Configuration Name
Enter the configuration name.
Service Logs
Task run logs are saved in the log topic cls_service_log to facilitate monitoring the task running status. This log topic is free and enabled by default.
Query statement
The PromQL statement to be executed. The pre-aggregation task will run this statement at scheduled intervals to retrieve the execution results.
Metric Name
The result of the execution statement will be stored under this metric name, which can be used for subsequent data queries. It supports English letters, numbers, underscores, and colons, and should conform to the regular expression [a-zA-Z_:][a-zA-Z0-9_:]*.
Custom Dimension
Adds dimensions to the metric. If there is a conflict between the custom dimension and the dimension names in the execution statement results, the custom dimension takes precedence.
Scheduling cycle
The execution interval for the pre-aggregation task, with the range from 1 to 1440 minutes. It is recommended to use a 1-minute interval.
Advanced Configurations
Target metric topic: Specifies where the pre-aggregated metric data will be stored. By default, it is stored in the current topic. If you want to store this data separately (e.g., to set a different retention period for this portion of data), you can store it in another metric topic.
Delayed execution: Since there may be delays in metric data collection, you can set a delayed execution to ensure the data is fully collected before the pre-aggregation task runs. The default delay is 30 seconds.
5. Once the configuration is complete, click Submit.

Import Configuration File

1. Log in to the Cloud Log Service console.
2. In the left sidebar, click Metric Topic.
3. Click the metric topic ID/name for which you want to create a pre-aggregation task to enter the metric topic management page.
4. Click the Metric Pre-aggregation tab to enter the pre-aggregation task list page, then click Import Configuration File. The main configuration items are described as follows:

Configuration Item

Description
Configuration Name
Enter the configuration name.
Enabling Status
Indicates whether the task needs to run. Tasks that are not running will not generate pre-aggregated result data.
Service Logs
Task run logs are saved in the log topic cls_service_log to facilitate monitoring the task running status. This log topic is free and enabled by default.
Execution interval
The execution interval for the pre-aggregation task, with the range from 1 to 1440 minutes. It is recommended to use a 1-minute interval.
YAML configuration
Compatible with Prometheus Recording Rule YAML. The interval only supports a range from 1 to 1440 minutes. For example:
groups:
  - name: example
    rules:
    - record: code:prometheus_http_requests_total:sum
      expr: sum by (code) (prometheus_http_requests_total)
Advanced Configurations
Target metric topic: Specifies where the pre-aggregated metric data will be stored. By default, it is stored in the current topic. If you want to store this data separately (e.g., to set a different retention period), you can store it in another metric topic.
Delayed execution: Since there may be delays in metric data collection, you can set a delayed execution to ensure the data is fully collected before the pre-aggregation task runs. The default delay is 30 seconds.
5. Click Submit to save the configuration file. A pre-aggregation task will be automatically generated, with the task name consisting of the group name and record from the YAML file.
Note:
Pre-aggregation tasks automatically generated through a configuration file should be modified via the configuration file; direct modification is not supported.

Manage the Configuration File

1. Log in to the Cloud Log Service console.
2. In the left sidebar, click Metric Topic.
3. Click the metric topic ID/name for which you want to create a pre-aggregation task to enter the metric topic management page.
4. Click the Metric Pre-aggregation tab to enter the pre-aggregation task list page, then click Manage the Configuration File.
5. In the list, you can view the existing configuration files. In the Operation column, you can manage the configuration files:
Edit: Edit the configuration file, with the configuration items consistent with Import Configuration File. After modification, the system will automatically adjust the pre-aggregation tasks based on the latest configuration file (deleting tasks that no longer exist in the file, adding new tasks, and updating existing tasks).
Pause: Suspend the pre-aggregation tasks associated with the configuration file. During the suspension, these tasks will no longer be executed.
Delete: Remove the pre-aggregation tasks associated with the configuration file.
次の記事へ: プライバシーポリシー

ヘルプとサポート

この記事はお役に立ちましたか？

フィードバック