tencent cloud

TDMQ

Tencent Distributed Message Queue (TDMQ)

Stable and reliable, high elasticity, low-cost cloud native message middleware.

Overview
Features
Product Family
Scenarios

Overview

Overview

Features

Product Family

Scenarios

Overview

Tencent Distributed Message Queue (TDMQ) is a series of message middleware products independently developed by Tencent Cloud. As a key component in distributed systems, it is stable and reliable, highly elastic, and low-cost, and provides basic capabilities of asynchronous communication. It reduces system complexity through application decoupling, and enhances system availability and scalability.

TDMQ is compatible with mainstream open-source protocols and provides five sub-products, including TDMQ for CKafka, TDMQ for RocketMQ, TDMQ for RabbitMQ, TDMQ for Apache Pulsar, and TDMQ for MQTT. It supports migration solutions with no business code modifications, reducing migration costs.

TDMQ covers online scenarios (such as e-commerce transactions and social live streaming), offline scenarios (such as big data real-time computing and offline analysis), and device-side scenarios (such as Internet of Things (IoT) and Internet of Vehicles (IoV)). It meets the needs of different industries and scenarios such as pan-internet, education, retail, transportation, finance, and healthcare.

Features
Out-of-the-Box and Ops-Free

TDMQ provides a fully managed service that is out-of-the-box. Users can create clusters with a few clicks, eliminating the need for tedious deployment. The comprehensive resource management interface, full-range monitoring metrics, and intelligent diagnosis tools significantly reduce Ops complexity and management costs.

Cross-AZ High Availability

TDMQ employs multiple technical measures to establish a comprehensive disaster recovery system. It adopts a cross-availability zone (AZ) deployment architecture to effectively mitigate IDC-level failure risks. Through traffic throttling protection policies, it dynamically adjusts traffic pressure to ensure cluster health. Meanwhile, it provides cross-cluster data replication capabilities to fully meet various high-availability scenario requirements, from basic disaster recovery to multi-site active-active deployment.

Rapid Scaling with High Elasticity

TDMQ provides premium elastic scaling capabilities, enabling rapid resource scaling with one-click operations. The underlying resource adjustment is seamless and transparent to businesses, easily handling various sudden traffic scenarios.

Serverless for Low Costs

TDMQ adopts a storage-compute separation architecture. The compute layer supports second-level elastic scaling, handling traffic surges without pre-provisioned resources to maximize resource utilization. In addition, the storage layer supports unlimited scalability with the pay-as-you-go billing mode, reducing storage costs by 30% to 50%.

Product Family
TDMQ for CKafka
TDMQ for CKafka is a distributed, high-throughput, highly scalable messaging system, 100% compatible with open-source Kafka. CKafka offers advantages such as high availability, data compression, and support for both offline and real-time data processing. It is suitable for scenarios such as log compression and collection, monitoring data aggregation, and streaming data integration.
Learn more
TDMQ for RocketMQ
TDMQ for RocketMQ is a distributed message middleware built on Apache RocketMQ, designed for message communication between distributed systems or components. It features massive message backlogs, low latency, high throughput, high reliability, and strong transaction consistency. This solution meets requirements for scenarios such as asynchronous decoupling, peak shifting, sequential message sending and receiving, distributed transaction consistency, and log synchronization.
Learn more
TDMQ for RabbitMQ
TDMQ for RabbitMQ is a message queue service independently developed by Tencent, supports the AMQP 0-9-1 protocol, is fully compatible with all components and concepts of open-source RabbitMQ, and offers advantages such as stability, security, and flexible scaling. It is commonly used for asynchronous communication between systems and service decoupling, reducing dependencies between different services.
Learn more
TDMQ for Apache Pulsar
TDMQ for Apache Pulsar is a message middleware developed by Tencent based on Apache Pulsar. It provides excellent cloud native and Serverless features, is compatible with all components and concepts of Apache Pulsar, and has underlying strengths of compute-storage separation and flexible scaling.
Learn more
TDMQ for MQTT
TDMQ for MQTT is a distributed and highly available message queue service. It adopts the "publish/subscribe" model to build an extremely lightweight messaging protocol. Compatible with the standard MQTT protocol, it supports seamless integration with open-source communities and mainstream MQTT clients without modifications. Enhancements have been made in stability, low latency, and high performance.
Learn more
Scenarios
  • Log analysis scenario has a strong demand for high throughput.
  • Both infrastructure and business applications generate a large volume of logs daily. Based on CKafka, you can rapidly build a log analysis and monitoring platform to promptly identify issues.
  • Upstream sends logs "in batches" "asynchronously" to the CKafka cluster to meet high throughput requirements.
  • Downstream systems such as Hadoop and Spark consume messages from CKafka to meet customers' offline business analysis needs.

  • E-commerce online transactions and live streaming have strong demands for high concurrency and low latency.
  • In scenarios such as large-scale promotions and flash sales, business traffic is high, and the difference in peak traffic from normal times can exceed a hundred times. RocketMQ supports rapid scaling, with peaks reaching up to million-level TPS, effectively handling business traffic peaks and meeting high concurrency demands.
  • For live streaming bullet comment interaction scenarios, real-time performance must be guaranteed. RocketMQ ensures low delay in massive message scenarios, ensuring real-time interactions during live streams and enhancing the customer business experience.
  • In IoV and IoT scenarios, there are large-scale device endpoints with strong demands for high concurrency requirements for both devices and messages.
  • MQTT supports bidirectional communication between devices and the cloud, enabling rapid delivery of device-side data to the server-side and real-time reporting of device status. It also supports issuing device commands for device-side operations.
  • MQTT supports high-concurrency, meeting the concurrency requirement for tens of millions of devices and messages.
  • When consuming messages, the importance of messages varies, and those with higher importance should be consumed first.
  • For example, in an order payment reminder scenario, when a customer places an order in the mall, the system promptly pushes the order notification to the customer. If payment is not made within the set time limit, an SMS reminder will be sent to the customer. Merchants categorize customers into key accounts and regular customers. For instance, payment reminders for key account orders require priority processing, while reminders for regular customers have relatively lower priority.
  • Using RabbitMQ's priority queue effectively supports this scenario, ensuring high-priority messages do not backlog for extended periods. If an order is identified as belonging to a key account, assign a relatively higher priority for prioritized processing; otherwise, apply the default priority.

Finance cross-regional disaster recovery scenarios usually have requirements for data reliability and cross-regional high availability.

  • Data reliability: Financial transactions have certain requirements for data reliability, strong consistency and throughput.
  • Cross-regional high availability: Financial customers typically need cross-regional disaster recovery capacity, support offsite disaster recovery switch so as to ensure business continuity and guarantee core business availability.

Pulsar solution:

  • Pulsar message data adopts Triple Data Replication Storage to improve data reliability, support multi-AZ deployment, and avoid service unavailability caused by single availability zone failure.
  • Based on the community GEO module, it supports cross-regional real-time message replication and one-click switching. Customer applications can support Cross-Region Disaster Recovery without code modification or configuration, and support proactive switchback after disaster recovery.
Contact Us

Click "Contact Us" to enjoy personalized presales consultation and aftersales service.