Operation Guide
Scenarios
This document describes how to use the Data Transfer Service (DTS) logical migration feature to migrate data from PostgreSQL to TencentDB for PostgreSQL.
Preparations
1. Read the Logical Migration Use Instructions carefully to learn about feature constraints and must-knows.
2. Establish the access channel between DTS and the database in advance according to the access type you want to use. For details, see Network Preparations.
Internet Data Center (IDC) self-built databases/Databases of other cloud vendors: The access method can be set to Public Network/DC/VPN Access/CCN.
During migration, ensure that the service of the source instance is accessible in the public network environment and maintain the stability of the public network connection. Network fluctuations or failures may cause the migration to fail. If a failure occurs, you need to re-initiate the migration task.
Self-built databases on Cloud Virtual Machine (CVM) instances: The access method can be set to Self-Built on CVM.
TencentDB instances: The access method can be set to Cloud Database.
3. The permission requirements for the source database are as follows:
When the source database is a PostgreSQL database other than TencentDB for PostgreSQL, the source database account must have the replication permission; otherwise, the pre-check will fail.
When the source database is a TencentDB for PostgreSQL database, the account of the source database must be the initialization user created when the cloud database instance is created.
If some tables or objects lack permissions, you can use a privileged account to execute the following example statements to grant permissions to each unauthorized object:
grant select on table table name to username;grant select on SEQUENCE sequence name to username;grant connect on database database name to username;grant select on large object large object OID to username;GRANT USAGE ON SCHEMA schema name to username;
4. The permission requirements for the target database are as follows.
The account for the target database must be the initialization user created when the cloud database instance is created.
If the target database instance contains the database to be migrated, but the owner of the database is not the migration user, you can execute the following statement to transfer the ownership of the database to the migration user:
alter database database name owner to migration user;
If the migration user (the account executing the migration task) is a non-
pg_tencentdb_superuser role user, an error message indicating that the target instance permission check fails and the schema list cannot be obtained will appear during checks. You can use the following statement to grant the permissions of the initialization user to the migration user.grant pg_tencentdb_superuser to migration user;
Operation Steps
1. (Optional) When you perform the full and incremental migration from PostgreSQL 9.4, 9.5, or 9.6 as the source database, refer to the following guide to install the tencent_decoding plugin. For other scenarios, this step can be skipped.
1.1 Download the plugin that matches the system architecture of the server hosting the source database.
Only system architectures of x86_64 and AArch64 are supported.
The plugin version needs to match the PostgreSQL version exactly.
The Glibc version must meet the following requirements: for x86_64 systems, it must be 2.17‑323 or later; for AArch64 systems, it must be 2.17‑260 or later.
Check the Glibc version on Linux systems:
rpm -q glibc
Check the Glibc version on other operating systems (Debian/Ubuntu/SUSE):
ldd --version | grep -i libc
Running Environment
Operating System: Ubuntu 24.04.2 LTS / x86_64
Runtime Version: GNU bash, version 5.2.21(1)-release (x86_64-pc-linux-gnu)
1.2 Place the downloaded tencent_decoding.so file in the lib folder under the Postgres process directory without restarting the instance.
2. Log in to the DTS console, select Data Migration in the left sidebar, and then click Create Migration Task to go to the migration task creation page.
3. On the migration task creation page, select the source instance type and region as well as the target instance type, region, and specification for migration, and then click Buy Now.
Configuration Item | Description |
Source Instance Type | Select the instance type based on your source database type. This setting cannot be changed after purchase. In this example, select PostgreSQL. |
Source Instance Region | Select the region where the source database is located. If the source database is self-built, select the region closest to it. |
Target Instance Type | Select the instance type based on your target database type. This setting cannot be changed after purchase. In this example, select PostgreSQL. |
Target Instance Region | Select the region where the target database is located. |
Specification | Select the specifications for the migration link based on your business needs. For performance and billing details of different specifications, see Billing Overview. |
4. On the page for setting the source and target databases, complete the settings for the task, the source database, and the target database. After the connectivity test between the source and target databases passes, click Save.
Note:
If the connectivity test fails, troubleshoot and resolve the issue according to the prompts and Troubleshooting Guide, and then try again.
Setting Type | Configuration Item | Description |
Task Configuration | Task Name | Specify a business-relevant name to facilitate task identification. |
| Running Mode | Immediate execution: The task will be started immediately once the task check passes. Scheduled execution: Configure when the task is executed. It will automatically start at the specified time. |
Source Database Settings | Source Database Type | Source database type selected during purchase. It cannot be modified. |
| Region | Source database region selected during purchase. It cannot be modified. |
| Access Type | Select the access type according to your scenario. For this example, set it to Cloud Database. For preparations related to different access types, see Preparation Overview. To ensure migration efficiency, cross-region migration is not supported for CVM self-built instances. If cross-region migration is required, select the public network access method. Public Network: The source database is accessible through a public IP address. During public network migration, ensure that the service of the source instance is accessible in the public network environment and maintain the stability of the public network connection. Network fluctuations or failures may cause the migration to fail. If a failure occurs, you need to re-initiate the migration task. Self-Build on CVM: The source database is deployed on a Tencent Cloud CVM instance. To ensure migration efficiency, cross-region migration is not supported for CVM self-built instances. If cross-region migration is required, select the public network access method. Direct Connect: The source database can connect to Tencent Cloud Virtual Private Cloud (VPC) through Direct Connect. VPN Access: The source database can connect to Tencent Cloud VPC through VPN Access. Database: The source database is a TencentDB instance. CCN: The source database can connect to Tencent Cloud VPC through CCN. |
| Cross-/Intra-Account | Intra-account: The Tencent Cloud root account for both the source and target database instances is the same. Cross-account: The Tencent Cloud root accounts for the source and target database instances are different. In this example, migration within the same account is performed. For guidance on cross-account migration operations, see Migration Between Cross-Account Cloud Database Instances. |
| Database Instance | Select the ID of the source PostgreSQL database instance. |
| Account | Select the database account for the source PostgreSQL database. The account permissions should meet the requirements. |
| Password | Specify the password for the source PostgreSQL database account. |
| Connection Method | Secure Sockets Layer (SSL) connection refers to the use of SSL to establish a secure connection between DTS and the database, encrypting the transfer link. Selecting the SSL connection may increase the database connection response time. Generally, the private network link of Tencent Cloud is relatively secure, and the SSL connection is not required to be enabled. However, for scenarios where transfer methods such as public networks and DC are adopted, and data security requirements are high, the SSL connection is required to be enabled. Before the SSL connection is selected, SSL encryption is required to be enabled for the database first. |
Target Database Settings | Target Database Type | Target database type selected during purchase. It cannot be modified. |
| Region | Target database region selected during purchase. It cannot be modified. |
| Access Type | Select the access type based on your scenario. For this example, set it to Database. |
| Database Instance | Select the ID of the target database. |
| Account | Target database account. The account permissions should meet the requirements. |
| Password | The password for the account of the target database. |
5. On the Set migration options and select migration objects page, configure the migration type and objects, and then click Next.
Configuration Item | Description |
Migration Type | Select the migration type based on your scenario. Structural migration: Structured data, such as databases and tables, will be migrated. Full migration: The entire database will be migrated. The migration includes only the data that exists in the source database when the task starts and excludes any new data written after the migration begins. Full + incremental migration: Both existing data in the source database at the start of the task and any new data written during the migration process are migrated. You can select this option if data is continuously written to the source database during migration, and a non-stop, hitless migration is required. |
Migration Object | Entire instance: The entire instance is migrated, including roles and user metadata definitions. System objects in postgres are excluded. Specify object: Migrate specified objects. Select the objects to be migrated from the source database objects and move them to the box of selected objects. |
Advanced Options | Limit Transfer Speed: Disabled by default. When enabled, it controls the data synchronization transfer speed. Concurrent Threads During Full Export: The full export phase consists of two steps: chunk calculation and chunk export. To reduce the total time of the full export phase, you can appropriately increase the number of threads. Note: Increasing the number of threads will increase the load on the source database. Source database full export requests per second (RPS): 0 Counts/s indicates no limit, and 50 million Counts/s is the maximum input limit. The actual export speed depends on various factors such as the source or target database load and network conditions. Concurrent Threads During Full Import: Increasing this number can improve the full write speed to the target database. Target database full import RPS: Increasing this import RPS can improve the full write speed to the target database. 0 Counts/s indicates no limit; 50 million Counts/s is the maximum input limit. The actual import speed depends on various factors such as source or target database load and network conditions. Full Export Traffic Limit of Source Database: Increasing the number can improve the speed of incremental imports. |
6. On the Verify task page, perform the check. After the check is completed and all check items are passed, click Immediate start.
7. If the check task fails, you can see Pre-check Failure Handling to fix the issue and re-initiate a check task.
Failed: indicates that a check item fails and the task is blocked. Fix the issue and execute the check task again.
Warning: indicates that a check item does not completely meet the requirements, and the task can be continued, but the business may be affected. Users are required to assess whether to ignore the warning or fix the issue before continuing to execute the task based on the prompt.
8. Return to the data migration task list. The task enters the Ready-to-run status. After running for 1 to 2 minutes, the data migration task will officially start.
Select Structural Migration or Full Migration: The task automatically ends upon completion and does not need to be manually ended.
Select Full + incremental migration: After full migration is completed, the task will automatically enter the incremental data synchronization phase. Incremental data synchronization does not stop automatically; you need to manually click Complete to end it. Once clicked, the task enters the Completing status. Do not modify any configurations at the source or target during this phase, as the backend will automatically align certain objects with the source.
Select an appropriate time to manually complete the incremental data synchronization and perform the service switch.
Once you confirm that the migration task is in the incremental synchronization phase and no latency exists, suspend the write operations on the source database for several minutes.
When the data gap between the target and source databases is 0 MB, and latency is 0 seconds, manually complete the incremental synchronization.
9. (Optional) If you need to view or delete tasks, click the task and perform the required operation in the Operation column. For details, see Managing Tasks.
10. When the status of the migration task changes to Task Successful, you can formally perform service cutover. For details, see Cutover Instructions.
Help and Support
Was this page helpful?
You can also Contact sales or Submit a Ticket for help.
Help us improve! Rate your documentation experience in 5 mins.
Feedback