TDMQ is a key component in the distributed architecture as it provides async communication and reduces system complexity through application decoupling to improve system availability and scalability. To meet the needs of different industries and scenarios in finance, internet, education, logistics, and energy, TDMQ offers a variety of product forms to cover both online and offline scenarios.
Tencent Distributed Message Queue (TDMQ)
TDMQ is a key component in the distributed architecture as it provides async communication and reduces system complexity through application decoupling to improve system availability and scalability. To meet the needs of different industries and scenarios in finance, internet, education, logistics, and energy, TDMQ offers a variety of product forms to cover both online and offline scenarios.
Overview
Features
Open-Source Component Compatibility
Offer a variety of product forms that are compatible with open source Kafka, RocketMQ, RabbitMQ, and Pulsar.
High Performance
Improve service performance to outperform open-source components.
High Availability
TDMQ not only supports the native Pulsar protocol but also is compatible with other popular message queue protocols that are integrated as plugins. You can migrate your messages to TDMQ with few code modifications.
High Reliability
Offer three data replicas and support automatic containerized restart in seconds without affecting message capacity and data.
Scalability
Scale elastically as needed and support processing tens of millions of high concurrent requests.
Security Control
Provide authentication, authorization, and root account/sub-account capabilities to enable enterprise-grade security protection.
Product Family
TDMQ for Pulsar
TDMQ for Pulsar is Tencent's proprietary message middleware based on Apache Pulsar. It comes with excellent cloud native and serverless features and is compatible with all components and principles of open-source Pulsar. It also has the underlying benefits of computing-storage separation and flexible scaling. It provides async decoupling and peak shifting capabilities for distributed application systems, and has the characteristics of massive message heap, high throughput, and reliable retry required by internet applications.
Learn moreTDMQ for CMQ
TDMQ for CMQ is a distributed message queue service that features a reliable message-based async communication mechanism. It enables message sending/receiving among different applications deployed in a distributed manner (or different components of the same application) and stores the delivered messages in reliable and valid message queues to prevent message loss. It supports multi-process simultaneous read/write, so that message sending and receiving do not interfere with each other, eliminating the need for the applications or components to keep running.
Learn moreTDMQ for RabbitMQ
TDMQ for RabbitMQ is a Tencent's proprietary message queue service that supports the AMQP 0-9-1 protocol and is fully compatible with all components and principles of Apache RabbitMQ. It has the advantages of stability, security, and flexible scaling. It is often used for async communication and service decoupling between systems to reduce the dependence between different services. It is widely used in distributed systems in industries such as finance, government affairs, retail, IoT, and social networking.
Learn moreTDMQ for RocketMQ
TDMQ for RocketMQ is a distributed message middleware based on Apache RocketMQ. It is applied to message communication between distributed systems or components. It has the characteristics of massive message heap, low latency, high throughput, high reliability, and strong transaction consistency, which meets the requirements of async decoupling, peak shifting, sequential sending and receiving, distributed transaction consistency, and log sync.
Learn moreCKafka
Message Queue CKafka (CKafka) is a high-throughput and highly scalable distributed messaging system that is fully compatible with open-source Kafka APIs 0.9-2.8 CKafka boasts the strengths such as high availability, data compression, and offline/real-time data processing, making it suitable for a variety of scenarios such as log compression collection, monitoring data aggregation, and streaming data aggregation.
Learn moreScenarios
Async Decoupling
Sequential Message Sending/Receiving
Peak Shifting
Consistency of Distributed Transactions
Log Aggregation
Website Tracking
Change Data Capture
IoT Scenarios
The transaction engine is the core system of Tencent billing. The data of each transaction order needs to be monitored by dozens of downstream business systems, including item price approval, delivery, reward point, and stream computing analysis. Such systems use different message processing logic, making it impossible for a single system to adapt to all associated business. In this case, TDMQ can implement efficient async communication and application decoupling to ensure the business continuity of the primary site.
Sequential messages are used in some business scenarios, such as order creation, payment, delivery, and refund of in-app/game items, which are all strictly executed in sequence. Similar to the First In, First Out (FIFO) principle, TDMQ offers a sequential message feature dedicated to such scenarios to ensure message FIFO.
Companies hold promotional campaigns such as new product launch and festival red packet grabbing from time to time, which often cause temporary traffic spikes and pose huge challenges to each backend application system. In this case, TDMQ can act as a buffer to centrally collect the suddenly increased requests in the upstream, allowing downstream businesses to consume the request messages based on their actual processing capacities.
A billing system often has a long transaction linkage with a significant chance of error or timeout. TDMQ's automated repush and abundant message retention features can be used to provide transaction compensation, and the eventual consistency of payment tips notifications and transaction pushes can also be achieved through TDMQ.
The log collection component collects and aggregates server business logs and system logs, forwards them to message queues for message persistence, and finally implements log archive, event retention, and big data analysis.
TDMQ collects website activity data in real time through message queues, and uses message flow for real-time monitoring and business analysis by subscribing to the real-time delivery of messages. It also supports loading to offline storage such as HDFS for analysis.
For structured data generated by business system, TDMQ captures the changed data through Debezium and delivers it to CKafka, which then transfers the data to ES, Spark, and Flink for further computing or processing. Data can be aggregated with ES to solve cross-table query performance issues. Then, the data can be sent to Spark and Flink for real-time data processing or synced between databases in a two-way manner via Kafka Stream or Connect.
The IoT devices deliver device data to data middleware through the MQTT protocol, and then distribute it to different systems through the rule engine for further processing.
Success Stories
China Literature
Honor of Kings
Tencent Midas
Contact Us
Click "Contact Us" to enjoy personalized presales consultation and aftersales service.