When using Kafka, you need to monitor its running status, such as cluster status and message heap. TMP provides an exporter to monitor Kafka and offers an out-of-the-box Grafana monitoring dashboard for it. This document describes how to deploy the Kafka exporter and integrate it with the alert feature.
Note:For easier export installation and management, we recommend you use TKE for unified management.
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
k8s-app: kafka-exporter # Rename the exporter based on the business needs. We recommend you add the Kafka instance information
name: kafak-exporter # Rename the exporter based on the business needs. We recommend you add the Kafka instance information
namespace: kafka-demo
spec:
replicas: 1
selector:
matchLabels:
k8s-app: kafka-exporter # Rename the exporter based on the business needs. We recommend you add the Kafka instance information
template:
metadata:
labels:
k8s-app: kafka-exporter # Rename the exporter based on the business needs. We recommend you add the Kafka instance information
spec:
containers:
- args:
- --kafka.server=x.x.x.x:9092 # Corresponding Kafka instance address information
image: danielqsj/kafka-exporter:latest
imagePullPolicy: IfNotPresent
name: kafka-exporter
ports:
- containerPort: 9121
name: metric-port # This name is required during scrape task configuration
securityContext:
privileged: false
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
imagePullSecrets:
- name: qcloudregistrykey
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
Note:For detailed exporter parameters, please see kafka_exporter.
Pod Monitor
to define a Prometheus scrape task. Below is a sample YAML configuration:apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
name: kafka-exporter # Enter a unique name
namespace: cm-prometheus # The namespace is fixed. Do not change it
spec:
podMetricsEndpoints:
- interval: 30s
port: metric-port # Enter the name of the corresponding port of the Prometheus exporter in the Pod YAML configuration file
path: /metrics # Enter the value of the corresponding path of the Prometheus exporter. If it is not specified, it will be `/metrics` by default
relabelings:
- action: replace
sourceLabels:
- instance
regex: (.*)
targetLabel: instance
replacement: 'ckafka-xxxxxx' # Change it to the corresponding Kafka instance ID
- action: replace
sourceLabels:
- instance
regex: (.*)
targetLabel: ip
replacement: '1.x.x.x' # Change it to the corresponding Kafka instance IP
namespaceSelector:
matchNames:
- kafka-demo
selector: # Enter the label value of the Pod to be monitored to locate the target Pod
matchLabels:
k8s-app: kafka-exporter
Note:As the exporter and Kafka are deployed on different servers, we recommend you use the Prometheus relabeling mechanism to add the Kafka instance information to the monitoring metrics so as to locate problems more easily.
Was this page helpful?