tencent cloud

Metrics Reporting
Download PDF
Last updated: 2024-09-20 17:48:27
Metrics Reporting
Last updated: 2024-09-20 17:48:27
Download PDF
Metric topics support the Prometheus Remote Write protocol, allowing various collectors compatible with this protocol to collect and report metrics to the metrics topics, such as vmagent and telegraf. With these collectors, you can report various types of monitoring metrics to the metric topics, such as:
Server monitoring metrics, such as CPU, memory, and disk utilization.
Container cluster monitoring metrics, such as Pods memory usage and apiserver request delay.
Middleware monitoring metrics, such as performance metrics for MySQL, Kafka, Nginx, and other middleware.
Application system monitoring metrics, such as API throughput, response time, and the number of error requests.
Metric topics can also serve as remote storage for a local Prometheus instance. Using the Remote Write method, metrics from the local Prometheus can be reported to the metric topics.

Reporting Address

Network Type
address
Public Network
https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write
Private Network
https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
Among them:
Replace ${region} with the region where the metric topic is located, such as ap-beijing. For more region abbreviations, see Available Regions. Currently, only the regions of Beijing, Shanghai, Guangzhou, and Nanjing are supported.
Replace ${topicId} with the metric topic ID, such as 0e69453c-0727-4c9c-xxxx-ea51b10d2aba. You can find the topic ID in the Metric Topic List.

Authentication method

Metric reporting uses Basic Auth for authentication. Use the SecretId and SecretKey from the API Key as the username and password, respectively.
username:${SecretId}
password:${SecretKey}
It is recommended to create a separate sub-account for the collector and use the SecretId and SecretKey of that account. Grant this sub-account only the following permissions to ensure security. For configuration details, see Sub-account Authorization.
{
    "version": "2.0",
    "statement": [
        {
            "effect": "allow",
            "action": [
                "cls:MetricsRemoteWrite"
            ],
            "resource": [
                "*"
            ]
        }
    ]
}

Configuration Example

Telegraf

Add the following output configuration to the Telegraf configuration file:
[[outputs.http]]
  ## Reporting address: Replace ${region} and ${topicId}. This example uses the public network address; if network conditions allow, it is recommended to use the private network address.
  ## Private network address URL = https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
  url = "https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write"

  ## Authentication information: Replace ${SecretId} and ${SecretKey}.
  username = "${SecretId}"
  password = "${SecretKey}"

  ## Do not modify the Telegraf output data format configuration.
  data_format = "prometheusremotewrite"
  [outputs.http.headers]
     Content-Type = "application/x-protobuf"
     Content-Encoding = "snappy"
     X-Prometheus-Remote-Write-Version = "0.1.0"

Vmagent

When starting Vmagent, use the following parameters to configure remoteWrite:
./vmagent-prod \\
  -remoteWrite.url=https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write \\
  -remoteWrite.basicAuth.username=${SecretId} \\
  -remoteWrite.basicAuth.password=${SecretKey}
Note:
The remoteWrite.url is set to the public network address; if network conditions allow, it is recommended to use the private network address.
The private network address is https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write.

Prometheus

Add the following configuration to the Prometheus configuration file:
# Reporting address: Replace ${region} and ${topicId}. This example uses the public network address; if network conditions allow, it is recommended to use the private network address.
# Private network address URL: https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
url: https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write

# Authentication information: Replace ${SecretId} and ${SecretKey}.
basic_auth:
  username: ${SecretId}
  password: ${SecretKey}

# Data write policy: Including caching and retry mechanisms, the following configuration is recommended for handling large data volumes.
queue_config:
  capacity: 20480
  min_shards: 100
  max_samples_per_send: 2048
  batch_send_deadline: 20s
  min_backoff: 100ms
  max_backoff: 5s
For more configuration details, see the official Prometheus documentation.

Prometheus Operator

Prometheus deployed in Kubernetes using the Prometheus Operator can report metrics in the following way:
1. Use kubectl to create the Secret required for authentication, for example, with the following command:
kubectl create secret generic kubepromsecret \\
 --from-literal=username=${SecretId} \\
 --from-literal=password=${SecretKey} \\
 -n monitoring
Note:
Replace ${SecretId} and ${SecretKey} in the command.
If Prometheus is deployed in another namespace, modify -n monitoring to the correct namespace.
You can also create Secret using a manifest file. For more details, see the official Kubernetes documentation.
2. Open the Prometheus manifest configuration file, which is typically located in the GitHub repo file used by the Prometheus Operator. The usual path is kube-prometheus/manifests/prometheus-prometheus.yaml.
3. Modify the Prometheus manifest configuration file by adding the following configuration at the end:
  remoteWrite:
  - url: "https://${region}.cls.tencentcs.com/prometheus/${topicId}/api/v1/write"
    basicAuth:
      username:
        name: kubepromsecret
        key: username
      password:
        name: kubepromsecret
        key: password
Note:
Replace ${region} and ${topicId} in the configuration. This example uses the public network address; if network conditions allow, it is recommended to use the private network address.
Private network address URL: https://${region}.cls.tencentyun.com/prometheus/${topicId}/api/v1/write
When handling large data volumes, you can add the following data write policy, including caching and retry mechanisms. For more details, see the official Prometheus Operator documentation.
    queueConfig:
      capacity: 204800
      minShards: 100
      maxShards: 2048
      maxSamplesPerSend: 4096
      batchSendDeadline: 30s
      minBackoff: 100ms
      maxBackoff: 5s
4. Apply the configuration file, for example, with the following command:
kubectl apply -f prometheus-prometheus.yaml -n monitoring
Note:
Replace prometheus-prometheus.yaml with the correct configuration file path.
If Prometheus is deployed in another namespace, modify -n monitoring to the correct namespace.

Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback