The data sync feature refers to real-time data sync between two database sources. It is suitable for cloud-local active-active, multi-site active-active, and cross-border data sync as well as real-time data warehousing.
Unlike data migration which is a one-time short-term task to migrate the entire database, data sync is a continuous task. After a task is created, the data will be continuously synced (almost in real time) to keep consistency between the source and target databases.
The following takes MySQL sync as an example to describe the data sync feature, where data is exported from the source database and imported into the target database, and key steps include structure initialization, full data initialization, and incremental data processing.
Structure initialization is to create the same table structure information in the target database as that in the source database. When configuring a sync task, you can select whether to sync the table structure. If the target database already contains the same structural information, you simply need to sync the data; otherwise, you'll need to sync the information as well.
Full data initialization
After structure initialization is completed, DTS will initialize the existing data, i.e., exporting all existing data from the source database and import it into the target database.
Incremental data processing
In incremental data processing, DTS gets the incremental data continuously through the source database's binlog and persistently stores such data in the intermediate storage after a series filtering and conversion operations. After importing the full data, it continuously replays the changed incremental data stored in the intermediate storage into the target database in order to ensure data consistency between the source and target databases.
The basic unit of the sync service is one-way sync. During configuration, you can choose to use data definition language (DDL) or data manipulation language (DML) for sync. By combining one-way sync tasks, you can customize various complex topologies.
In a complex topology, technical measures will be used for DML operations to avoid data loop. However, for DDL, the data sync service will check for loop during configuration to avoid forming DDL loop.
Below are some common topologies, which you can customize by purchasing multiple sync instances:
By using DTS, you can sync data between MySQL databases in multiple regions to implement multi-site active-active deployment. Database instances in each region can run in the cloud or your self-built IDC.
You can implement two-way sync by combining one-way sync tasks, but the following restrictions exist:
For more information on the source and target database types, versions, and sync types supported for data sync, see Databases Supported by Data Sync.
|Two-way sync, ring sync, and many-to-one sync||Complex sync topologies such as two-way sync, ring sync, and many-to-one sync are supported.||Databases Supported by Data Sync|
|Cross-account sync||Data sync between instances under different Tencent Cloud accounts is supported.||Cross-account TencentDB Instance Sync|
|Cross-version sync of most databases||The target database version should be equal to or later than the source database version; for example, data on v5.5.x can be synced to v5.5.x, v5.6.x, or later. The last digit in the version number is the minor version number, which is not restricted.||-|
|Table conflict check||The duplicate table name check policy is provided.||-|
|Primary key conflict check||The following three primary key conflict processing policies are supported:
|DML and DDL filtering||
||Setting SQL Sync Policy|
|WHERE conditional filter||You can customize a filter for a single table.||Setting SQL Sync Policy|
|Sync of views and advanced objects||Views, stored procedures, functions, triggers, and events can be synced.||-|
|Task progress visualization||Information such as sync steps and progress can be displayed.||-|
|Metric monitoring and default alarm policy||Supported Events and Metrics|
|Database restart or upgrade||During incremental data sync, the source and target databases can be restarted or upgraded.||-|