tencent cloud

Cloud Log Service

DocumentationCloud Log ServiceOperation GuideShipping and ConsumptionLog ConsumptionConsumption over KafkaConsumption Parameter Description

Consumption Parameter Description

PDF
Focus Mode
Font Size
Last updated: 2025-12-03 18:30:48
The following consumer parameters support customizable configuration, but we recommend using default values as a priority. If adjustments are needed, ensure cautious evaluation of the impact to avoid consumption is abnormal or performance issues caused by misconfiguration.

Common Consumption Parameters Overview

The following are some common consumption parameters.

Parameters for Offset Management

Parameter Name
Default Value
Description
auto.offset.reset
latest
Earliest: Automatically reset to the earliest offset.
Latest: Automatically reset to the latest offset.
enable.auto.commit
true
If true, the consumer offset will be submitted periodically in the backend.
auto.commit.interval.ms
5000 ms
If enable.auto.commit is set to true, the frequency of automatic commit for consumer offset (ms).

Performance Adjustment Parameters

Warning:
It is recommended to use default values. Operate with caution when adjusting the following parameters. Improper adjustment can cause data backlog, redundant requests and other issues.
Parameter Name
Default Value
Description
fetch.max.wait.ms
500 ms
Maximum waiting time for consumers to pull messages
fetch.min.bytes
1MB
The maximum data volume a server returns for a single request. If there is no sufficient data, the request will wait.
fetch.max.bytes
50MB
The maximum data volume a server returns for a single request.
Too small (such as 1M): The amount of data is small every time, requiring more requests to obtain sufficient data, increasing the number of sessions on the server.
Too large (such as 50M+): May exceed the client's processing capability, causing processing timeout, consumption backlog, and in extreme cases, repeatedly request the same batch of data from the server, leading to an increase in your metered billing.
request.timeout.ms
30000 ms
Request timeout period. Use together with fetch.max.bytes.
Too short (such as 5s): May cause batch processing to return before reaching fetch.max.bytes, reducing processing efficiency.
Too long (such as 60s): Increases message processing latency.
max.poll.records
5000
Maximum record count returned by a single poll() call.
session.timeout.ms
10000 ms
Consumer session timeout period with Kafka server (ms).
heartbeat.interval.ms
3000 ms
Consumer heartbeat interval (ms).

Parameters for Fault Recovery

Parameter Name
Default Value
Description
reconnect.backoff.ms
50 ms
Initial backoff time to reconnect to server (ms).
retry.backoff.ms
100 ms
Waiting time before retry failure (ms).
max.poll.interval.ms
120000 ms
The upper limit between two poll() calls. If exceeded, the consumer will be considered failed and trigger a rebalance.

Adjusting Parameters Verification Method

Introduce a simple and direct solution to verify whether your adjusted consumption parameters are reasonable and valid. By monitoring the consumption delay (Latency) metric (divided into log topic/partition granularity), you can judge whether current consumption is normal.
If the value of the indicator keeps increasing, it means your consumer end parameters are unreasonable and need to be readjusted.
If the metric is small (less than 3 seconds) and remains unchanged, it is normal.
Consumption delay means the consumption speed lags behind production speed, which can lead to data backlog. Recommendation:
Configure a consumer for each consumption partition to expand the consumer end's ability.
Configure fetch.max.bytes not too large to reduce data returned by server.


Previous Topic: Task Monitoring and AlarmNext Topic: Consumption Demo - multi-language SDK

Help and Support

Was this page helpful?

Help us improve! Rate your documentation experience in 5 mins.

Feedback