Data subscription refers to the process where DTS gets the data change information of a key business in the database, converts it into message objects, and pushes them to Kafka for the downstream businesses to subscribe to, get, and consume. DTS allows you to directly consume data through a Kafka client, so you can build data sync features between TencentDB databases and heterogeneous systems, such as cache update, real-time ETL (data warehousing technology) sync, and async business decoupling.
The following takes MySQL as an example to describe how data subscription pulls the incremental binlog from the source database in real time, parses the incremental data into Kafka messages, and then stores them on the Kafka server. You can consume the data through a Kafka client. As an open-source messaging middleware, Kafka supports multi-channel data consumption and SDKs for multiple programming languages to reduce your use costs.
By using the data subscription feature of DTS, you can push the updated incremental data in TencentDB to an archive database or data warehouse as a stream in real time.
DTS allows you to subscribe to databases and tables. Specifically, the following three subscription types are supported:
|SDKs for various programming languages||DTS uses the Kafka protocol and supports Kafka client SDKs for multiple programming languages.||-|
|Metric monitoring and default alarm policy||Supported Events and Metrics|
|Multi-Channel data consumption||DTS allows creating multiple data channels for a single database, which can be consumed concurrently through a consumer group.||-|
|Partitioned consumption||DTS supports partitioned storage of data in a single topic for concurrent consumption of data in multiple partitions, improving the consumption efficiency.||-|