tencent cloud

Throttling
Last updated: 2025-07-15 17:46:18
Throttling
Last updated: 2025-07-15 17:46:18

Overview

TDMQ for RocketMQ provides message service for various large-scale, low-latency online services with high availability requirements. Clients establish persistent connections with RocketMQ clusters via SDK to send and receive messages, consuming compute, storage, and network bandwidth resources of cluster machines. To deliver high-quality message service, we need to control the load watermarks of clusters under high-concurrency and heavy traffic to ensure system stability and reliability. Therefore, the server limits the maximum converted message count per second (Transaction Per Second/TPS) that clients can send and consume based on cluster specifications. For specific calculation rules, see Billing Overview - Compute Specification. To balance isolation and flexibility, TPS quotas for sending and consuming messages are not shared, while supporting customizable quota ratios (default ratio is 1:1).


Throttling Behavior Description

TDMQ for RocketMQ implements a quick failure (Fail-Fast) throttling policy, where when client request rate reaches the limit, the server immediately responds with an error. Normally, online services are sensitive to response time. Quick failure enables clients to detect throttling events and intervene promptly, preventing prolonged end-to-end message duration in business operations.
For a base cluster with a 1000 TPS specification, assume the send and receive TPS quota ratio is 1:1. The related throttling trigger behavior description is as follows:
Description
Message Throttling for Sending
Message Throttling for Consumption
Throttling Trigger Scenario
All connections to the cluster can send a total of 500 converted messages per second. Requests will fail if the sending rate reaches the limit.
All consumer clients connected to the cluster can consume a total of 500 converted messages per second. Message consumption delay will increase if the consumption rate reaches the limit.
Log Keyword When Trigger Throttling
Rate of message sending reaches limit, please take a control or upgrade the resource specification.
Rate of message receiving reaches limit, please take a control or upgrade the resource specification.
Retry Mechanism When Trigger Throttling
SDKs handle differences across protocols:
5.x SDK retries sending with exponential backoff policy. The maximum number of retries can be customized during Producer initialization, with a default value of 2. Failed requests that reach the maximum retry limit will throw an exception.
4.x SDK directly throws exception and will not retry.
The SDK pull message thread automatically performs backoff retry.

Client practical tutorial

Planning a Cluster

The purpose of cluster rate limiting in TDMQ for RocketMQ is to ensure service stability and reliability, prevent issues such as increased service response time and decreased request success rate in case of high load, thereby avoiding business impact. Therefore, when integrating TDMQ for RocketMQ, reasonably planning the cluster is very important. We recommend that you:
Assess business TPS fully based on current scale and future trend prediction. If business traffic fluctuates, use peak TPS as the benchmark. In addition, we recommend reserving some TPS quota (such as 30%) to handle possible traffic spikes.
For businesses with high stability requirements, we recommend using multiple RocketMQ clusters to enhance isolation. For example, isolate core links (such as transaction systems) from non-core links (such as log systems), and separate production environments from development and testing environments.

Monitoring Workload

You can use the monitoring and alerting capabilities of the TDMQ for RocketMQ console to observe cluster load in real-time, discover TPS water level risks in advance, and perform timely upgrades to ensure adequate resources and avoid triggering throttling. For specific operations, see Monitoring Alarm. Alarm policy recommendations are as follows:
Send and consumption TPS water level exceeding 70% of capacity triggers alarm to remind perform upgrade estimation.
Trigger alarm when traffic throttling occurs, warning that business sending may experience failures.

Example

For a base cluster with 1000 TPS specification, the TPS alarm policy is as follows:


Handling Code Exceptions

When sending a message via the RocketMQ SDK, the business code must capture exceptions including traffic throttling errors and save necessary context information for human intervention to resume business. Retry mechanisms vary across SDKs of different protocols. Sample code for related handling:
4.x SDK does not automatically retry traffic throttling errors, so business code must catch exception and handle it. Sample code:
import java.io.UnsupportedEncodingException;
import java.nio.charset.StandardCharsets;
import org.apache.rocketmq.acl.common.AclClientRPCHook;
import org.apache.rocketmq.acl.common.SessionCredentials;
import org.apache.rocketmq.client.exception.MQBrokerException;
import org.apache.rocketmq.client.exception.MQClientException;
import org.apache.rocketmq.client.producer.DefaultMQProducer;
import org.apache.rocketmq.client.producer.SendResult;
import org.apache.rocketmq.common.message.Message;
import org.apache.rocketmq.common.message.MessageClientIDSetter;
import org.apache.rocketmq.logging.InternalLogger;
import org.apache.rocketmq.logging.InternalLoggerFactory;

public class ProducerExample {
private static final InternalLogger log = InternalLoggerFactory.getLogger(ProducerExample.class);

public static void main(
String[] args) throws MQClientException, InterruptedException, UnsupportedEncodingException {

String nameserver = "Your_Nameserver";
String accessKey = "Your_Access_Key";
String secretKey = "Your_Secret_Key";
String topicName = "Your_Topic_Name";
String producerGroupName = "Your_Producer_Group_Name";

// Instantiate the message producer Producer
DefaultMQProducer producer = new DefaultMQProducer(
producerGroupName, // Producer group name
new AclClientRPCHook(new SessionCredentials(accessKey, secretKey)) // ACL permission, accessKey and secretKey can be obtained on the console cluster permission page
);

// Set Nameserver address, can be obtained on the console cluster basic information page
producer.setNamesrvAddr(nameserver);

Start the Producer instance
producer.start();

// Create message instance, set topic and message content
Message message = new Message(topicName, "Your_Biz_Body".getBytes(StandardCharsets.UTF_8));

// Maximum send count, set based on business needs
final int maxAttempts = 3;
// retry interval, set based on business needs
final int retryIntervalMillis = 200;

// Send the message.
int attempt = 0;
do {
try {
SendResult sendResult = producer.send(message);
log.info("Send message successfully, {}", sendResult);
break;
} catch (Throwable t) {
attempt++;
if (attempt >= maxAttempts) {
// Reached the maximum number of times
log.warn("Failed to send message finally, run out of attempt times, attempt={}, maxAttempts={}, msgId={}",
attempt, maxAttempts, MessageClientIDSetter.getUniqID(message), t);
// Log messages that failed to send (or record in other services, such as database etc.)
log.warn(message.toString());
break;
}
int waitMillis;
if (t instanceof MQBrokerException && ((MQBrokerException) t).getResponseCode() == 215 /* FLOW_CONTROL */) {
// Rate limit exceeded, use backoff retry
waitMillis = (int) Math.pow(2, attempt - 1) * retryIntervalMillis; // Retry interval: 200ms, 400ms, ......
} else {
other exceptions
waitMillis = retryIntervalMillis;
}
log.warn("Failed to send message, will retry after {}ms, attempt={}, maxAttempts={}, msgId={}",
waitMillis, attempt, maxAttempts, MessageClientIDSetter.getUniqID(message), t);
try {
Thread.sleep(waitMillis);
} catch (InterruptedException ignore) {
}
}
}
while (true);

producer.shutdown();
}
}
5.x SDK automatically retries sending exceptions. The business code can customize the maximum number of retries. Sample code:
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import org.apache.rocketmq.client.apis.ClientConfiguration;
import org.apache.rocketmq.client.apis.ClientException;
import org.apache.rocketmq.client.apis.ClientServiceProvider;
import org.apache.rocketmq.client.apis.SessionCredentialsProvider;
import org.apache.rocketmq.client.apis.StaticSessionCredentialsProvider;
import org.apache.rocketmq.client.apis.message.Message;
import org.apache.rocketmq.client.apis.producer.Producer;
import org.apache.rocketmq.client.apis.producer.SendReceipt;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class ProducerExample {
private static final Logger log = LoggerFactory.getLogger(ProducerExample.class);

public static void main(String[] args) throws ClientException, IOException {

String nameserver = "Your_Nameserver";
String accessKey = "Your_Access_Key";
String secretKey = "Your_Secret_Key";
String topicName = "Your_Topic_Name";

// ACL permission, accessKey and secretKey can be obtained on the console cluster permission page
SessionCredentialsProvider sessionCredentialsProvider = new StaticSessionCredentialsProvider(accessKey, secretKey);

ClientConfiguration clientConfiguration = ClientConfiguration.newBuilder()
.setEndpoints(nameserver) // Set NameServer address, can be obtained on the console cluster basic information page
.setCredentialProvider(sessionCredentialsProvider)
.build();

Start the Producer instance
ClientServiceProvider provider = ClientServiceProvider.loadService();
Producer producer = provider.newProducerBuilder()
.setClientConfiguration(clientConfiguration)
.setTopics(topicName) // Pre-declare the topic for message sending, recommend setting
.setMaxAttempts(3) // Maximum send count, set based on business needs
.build();

// Create message instance, set topic and message content
byte[] body = "Your_Biz_Body".getBytes(StandardCharsets.UTF_8);
final Message message = provider.newMessageBuilder()
.setTopic(topicName)
.setBody(body)
.build();

try {
final SendReceipt sendReceipt = producer.send(message);
log.info("Send message successfully, messageId={}", sendReceipt.getMessageId());
} catch (Throwable t) {
log.warn("Failed to send message", t);
// Log messages that failed to send (or record in other services, such as databases)
log.warn(message.toString());
}

producer.close();
}
}

FAQs

Trigger Throttling and Message Loss

When message sending triggers throttling, the server does not store the message. The client must catch the exception and perform downgrade processing. If consumption triggers throttling, a delay in message consumption occurs, but successfully sent messages are not lost.

Why Is TPS on Monitoring Page Higher Than Message Count

TPS converts message count. If business operations use advanced messages (sequential, transactional, delayed, etc.) or the message body is relatively large, one business message will be counted as multiple converted messages. For specific conversion logic, see Billable Items. In addition, the message count metric calculates the per-second mean within one minute, while the TPS metric calculates the per-second peak value within one minute.

Cluster Occasionally Experiences Brief Rate Limited Consumption, Is There Any Impact

Generally no impact. During operations such as client restart, server restart, or console expansion of topic queues, temporary consumption throttling may be triggered due to accumulation in the consumption group. It usually recovers quickly after stabilization.

How to Determine Whether Cluster Traffic Throttling Is Displayed

In addition to identifying exceptions thrown by the SDK Sending Interface or logging info from the SDK, you can also check the Tencent Cloud RocketMQ Console > Monitoring Dashboard for rate-limited production TPS (Count/s) and rate-limited consumption TPS (Count/s).


Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback