app_id, migratejob_id; data sync: appid, replicationjobid; data subscription: appId, subscribeid)Metric Category | Metric Name | Parameter | Unit | Description |
BPS – Full Stage | Full Export BPS | MigrateDumperBps | MB/s | Volume of data exported from the source instance per second in the full stage |
| Full Import BPS | MigrateLoaderBps | MB/s | Volume of data imported to the target instance per second in the full stage |
BPS – Incremental Stage | Incremental Export BPS | MigrateRiverBps | MB/s | Volume of data exported from the source instance per second in the incremental stage |
| Incremental Import BPS | MigrateSinkerBps | MB/s | Volume of data imported to the target instance per second in the incremental stage |
RPS – Full Stage | Full Export RPS | MigrateDumperRps | Count/s | Number of rows of data exported from the source instance per second in the full stage |
| Full Import RPS | MigrateLoaderRps | Count/s | Number of rows of data imported to the target instance per second in the full stage |
RPS – incremental stage | Source Instance Data Extraction RPS (Incremental Export RPS) | MigrateCaptureRps | Count/s | This metric is only used in the transitory stage, and the "Incremental Export RPS" metric will be finally used. |
| Incremental Export RPS | MigrateRiverRps | Count/s | Number of rows of data exported from the source instance per second in the incremental stage |
| Target Instance Data Loading RPS (Incremental Import RPS) | MigrateLoadRps | Count/s | This metric is only used in the transitory stage, and the "Incremental Import RPS" metric will be finally used. |
| Incremental Import RPS | MigrateSinkerRps | Count/s | Number of rows of data imported to the target instance per second in the incremental stage |
Network Latency – Full Stage | Source Instance Network Latency During Full Export | MigrateDumperNetworkLag | ms | Network latency between data export and source instance in the full stage |
| Target Instance Network Latency During Full Import | MigrateLoaderNetworkLag | ms | Network latency between data import and target instance in the full stage |
Network Latency – Incremental Stage | Source Instance Network Latency During Incremental Export | MigrateRiverNetworkLag | ms | Network latency between data export and source instance in the incremental stage |
| Target Instance Network Latency During Incremental Import | MigrateSinkerNetworkLag | ms | Network latency between data import and target instance in the incremental stage |
| Data Migration Time Lag | MigrateLag | s | Time lag between source and target instances in the incremental stage Calculation method: Time lag = the current time of the source instance - the time recorded when the latest source instance binlog event is being executed in the target instance. In scenarios where the source database is read-only, a time lag occurs if no data is written to the source database for a long time. The calculation of the data migration time lag depends on the incremental binlogs of the source database. Therefore, if no DDL or DML operation is performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual migration time lag. For example, the value -1 indicates that the existing data has been migrated, but no incremental data is updated. To solve this issue, DTS adopts the heartbeat mechanism in the source database to continuously update the incremental data. However, this mechanism is unavailable if the source database is read-only. As a result, a time lag still exists. |
| Data Gap in Data Migration | MigrateLagData | MBytes | Data gap between the source and target instances in the incremental stage Calculation method: Data gap = the file offset of the latest source instance binlog event - the file offset of the latest source instance binlog event which is being executed in the target instance. If the two offsets are in different binlog files, this value will be an estimate. In scenarios where the source database is read-only, a time lag occurs if no data is written to the source database for a long time. For example, the value -1 indicates that the existing data has been migrated, but no incremental data is updated. |
RPS Utilization | RPS Utilization During Incremental Import | MigrateSinkerRpsUsage | % | RPS utilization = real-time RPS/maximum RPS. The maximum RPS depends on the specification you select when configuring the link. When the RPS utilization almost reaches 100%, the incremental migration can no longer be accelerated. You need to upgrade the specification for faster migration speed. When the link is configured with the highest specification, there is no RPS limit, and the RPS may exceed 100%. |
Metric Category | Metric Name | Parameter | Unit | Description |
BPS – Full Stage | Source Instance Full Export BPS | MigrateDumperBps | MB/s | Volume of data exported by DTS from the source instance per second in the full stage. |
| Target Instance Full Import BPS | MigrateLoaderBps | MB/s | Volume of data imported by DTS into the target instance per second in the full stage. |
BPS – Incremental Stage | Source Instance Incremental Export BPS | MigrateRiverBps | MB/s | Volume of data exported by DTS from the source instance per second in the incremental stage. |
| Target Instance Incremental Import BPS | MigrateSinkerBps | MB/s | Volume of data imported by DTS into the target instance per second in the incremental stage. |
RPS – Full Stage | Target Instance Full Import RPS (Number of Commands per Second) | MigrateLoaderRps | Count/s | Number of commands for importing data into the target database per second in the full stage. |
RPS – Incremental Stage | Source Instance Incremental Export RPS (Number of Commands per Second) | MigrateRiverRps | Count/s | Number of commands for exporting data from the source database per second in the incremental stage. Initial value: 0. The value is refreshed after the complete file is received. |
| Target Instance Incremental Import RPS (Number of Commands per Second) | MigrateSinkerRps | Count/s | Number of commands for importing data into the target database per second in the incremental stage. Initial value: 0. The value is refreshed after any of the shards enters the incremental stage. |
Network Latency – Full Stage | Source Instance Network Latency During DTS Full Export | MigrateDumperNetworkLag | ms | Network latency of the source instance in the DTS full stage. |
| Target Instance Network Latency During DTS Full Import | MigrateLoaderNetworkLag | ms | Network latency of the target instance in the DTS full stage. |
Network Latency – Incremental Stage | Source Instance Network Latency During DTS Incremental Export | MigrateRiverNetworkLag | ms | Network latency of the source instance in the DTS incremental stage. |
| Target Instance Network Latency During DTS Incremental Import | MigrateSinkerNetworkLag | ms | Network latency of the target instance in the DTS incremental stage. |
| Data Migration Time Lag | MigrateLag | s | Time lag between the source and target instances. Calculation method: Volume of delayed incremental data/current incremental import BPS of the target instance. The value is refreshed after any of the shards enters the incremental stage. |
| Data Volume Gap in Data Migration | MigrateLagData | MBytes | Data gap between the source and target instances. Calculation method: Difference between the repl_offset value of the source instance and the volume of data imported by DTS The value is refreshed after any of the shards enters the incremental stage. |
Metric Category | Metric Name | Parameter | Unit | Description |
BPS – Full Stage | Source Instance Full Export BPS | MigrateDumperBps | MB/s | Volume of data exported by DTS from the source instance per second in the full stage. |
| Target Instance Full Import BPS | MigrateLoaderBps | MB/s | Volume of data imported by DTS into the target instance per second in the full stage. |
BPS – Incremental Stage | Source Instance Incremental Export BPS | MigrateRiverBps | MB/s | Volume of data exported by DTS from the source instance per second in the incremental stage. |
| Target Instance Incremental Import BPS | MigrateSinkerBps | MB/s | Volume of data imported by DTS into the target instance per second in the incremental stage. |
RPS – Full Stage | Source Instance Full Export RPS | MigrateDumperRps | Count/s | Number of documents exported by DTS from the source instance per second in the full stage. |
| Target Instance Full Import RPS | MigrateLoaderRps | Count/s | Number of documents imported by DTS into the target instance per second in the full stage. |
RPS – Incremental Stage | Source Instance Incremental Export RPS | MigrateRiverRps | Count/s | Number of documents exported by DTS from the source instance per second in the incremental stage. The initial value is "0". The value is refreshed after the incremental phase is entered, and the source instance generates new logs. |
| Target Instance Incremental Import RPS | MigrateSinkerRps | Count/s | Number of documents imported by DTS into the target instance per second in the incremental stage. The initial value is "0". The value is refreshed after the incremental phase is entered, and the source instance generates new logs. |
Network Latency – Full Stage | Source Instance Network Latency During DTS Full Export | MigrateDumperNetworkLag | ms | Network latency of the source instance in the DTS full stage. |
| Target Instance Network Latency During DTS Full Import | MigrateLoaderNetworkLag | ms | Network latency of the target instance in the DTS full stage. |
Network Latency – Incremental Stage | Source Instance Network Latency During DTS Incremental Export | MigrateRiverNetworkLag | ms | Network latency of the source instance in the DTS incremental stage. |
| Target Instance Network Latency During DTS Incremental Import | MigrateSinkerNetworkLag | ms | Network latency of the target instance in the DTS incremental stage. |
| Data Migration Time Lag | MigrateLag | s | Time lag between the source and target instances. Calculation method: The current time of the source instance minus the time recorded in the latest source instance oplog or change stream event that is currently being executed. The initial value is "-1". The value is refreshed after the incremental phase is entered and the source instance generates new logs. |
Metric Category | Metric Name | Parameter | Unit | Description |
BPS – Full Stage | Full Export BPS | ReplicationDumperBps | MB/s | Volume of data exported from the source instance per second in the full stage |
| Full Import BPS | ReplicationLoaderBps | MB/s | Volume of data imported to the target instance per second in the full stage |
BPS – Incremental Stage | Incremental Export BPS | ReplicationRiverBps | MB/s | Volume of data exported from the source instance per second in the incremental stage |
| Incremental Import BPS | ReplicationSinkerBps | MB/s | Volume of data imported to the target instance per second in the incremental stage |
RPS – Full Stage | Full Export RPS | ReplicationDumperRps | Count/s | Number of rows of data exported from the source instance per second in the full stage |
| Full Import RPS | ReplicationLoaderRps | Count/s | Number of rows of data imported to the target instance per second in the full stage |
RPS – incremental stage | Source Instance Data Extraction RPS (Incremental Export RPS) | ReplicationCaptureRps | Count/s | This metric is only used in the transitory stage, and the "Incremental Export RPS" metric will be finally used. |
| Incremental Export RPS | ReplicationRiverRps | Count/s | Number of rows of data exported from the source instance per second in the incremental stage |
| Target Instance Data Loading RPS (Incremental Import RPS) | ReplicationLoadRps | Count/s | This metric is only used in the transitory stage, and the "Incremental Import RPS" metric will be finally used. |
| Incremental Import RPS | ReplicationSinkerRps | Count/s | Number of rows of data imported to the target instance per second in the incremental stage |
Network Latency – Full Stage | Source Instance Network Latency During Full Export | ReplicationDumperNetworkLag | ms | Network latency between data export and source instance in the full stage |
| Target Instance Network Latency During Full Import | ReplicationLoaderNetworkLag | ms | Network latency between data import and target instance in the full stage |
Network Latency – Incremental Stage | Source Instance Network Latency During Incremental Export | ReplicationRiverNetworkLag | ms | Network latency between data export and source instance in the incremental stage |
| Target Instance Network Latency During Incremental Import | ReplicationSinkerNetworkLag | ms | Network latency between data import and target instance in the incremental stage |
| Data Sync Time Lag | DtsReplicationLag | s | Sync time lag between source and target instances in the incremental stage Calculation method: Time lag = the current time of the source instance - the time recorded when the latest source instance binlog event is being executed in the target instance. In scenarios where the source database is read-only, a time lag occurs if no data is written to the source database for a long time. The calculation of the data sync time lag depends on the incremental binlogs of the source database. Therefore, if no DDL or DML operation is performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync time lag. For example, the value -1 indicates that the existing data has been migrated, but no incremental data is updated. To solve this issue, DTS adopts the heartbeat mechanism in the source database to continuously update the incremental data. However, this mechanism is unavailable if the source database is read-only. As a result, a time lag still exists. |
| Data Gap in Data Sync |
DtsReplicationLagData | MBytes | Sync data gap between the source and target instances in the incremental stage. Calculation method: Data gap = the file offset of the latest source instance binlog event - the file offset of the latest source instance binlog event which is being executed in the target instance. If the two offsets are in different binlog files, this value will be an estimate. In scenarios where the source database is read-only, a time lag occurs if no data is written to the source database for a long time. For example, the value -1 indicates that the existing data has been migrated, but no incremental data is updated. |
RPS utilization | RPS Utilization During Incremental Import | ReplicationSinkerRpsUsage | % | RPS utilization = real-time RPS/maximum RPS. The maximum RPS depends on the specification you select when configuring the link. When the RPS utilization almost reaches 100%, the incremental migration can no longer be accelerated. You need to upgrade the specification for faster migration speed. When the link is configured with the highest specification, there is no RPS limit, and the RPS may exceed 100%. |
Metric Category | Metric Name | Parameter | Unit | Description |
Data Production |
GTID Quantity Gap Between Subscription Service and Source Database | ProducerLag | Count | Gap between the number of GTIDs in the binlog event already parsed by the data subscription service and the number of GTIDs in the latest binlog event generated by the source database. If there are no DDL or DML operations performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync time lag. If the value is -1, it indicates that there is no refresh of the incremental data. |
| Number of Transactions Parsed per Second | ProducerTps | Count/s | Number of transactions parsed from the source instance binlog per second in the incremental stage |
| Incremental Export BPS | SubscribeRiverBps | MB/s | Volume of data exported from the source instance per second in the incremental stage |
| Incremental Import BPS to Kafka | SubscribeSinkerBps | MB/s | Volume of data imported to the built-in Kafka of the subscription service per second in the incremental stage |
Data Consumption | Subscription Partition Consumption Gap | SubscribePartitionLag | Count | Gap between the offset of data to be consumed and that of consumed data. The statistical dimensions are "Partition", "Group", and "Task". |
| Subscription Partition Consumption Time Lag | SubscribePartitionLagTime | s | Time lag between the consumed data and the source instance. The statistical dimensions are "Partition", "Group", and "Task". |
Metric Category | Metric Name | Parameter | Unit | Description |
Data Production | GTID Quantity Gap Between Subscription Service and Source Database | ProducerLag | Count | Gap between the number of GTIDs in the binlog event already parsed by the data subscription service and the number of GTIDs in the latest binlog event generated by the source database. If no DDL or DML operation is performed on the source database for a long time, the value of this metric will gradually increase and thus cannot reflect the actual sync time lag. For example, the value -1 indicates that no data is updated in the source database. |
| Number of Transactions Parsed per Second | ProducerTps | Count/s | Number of transactions extracted and parsed by DTS from the source instance binlog per second. |
Data Consumption | Number of Records Not Consumed by Subscription Partition | SubscribeConsumePartitionUnConsume | Count | Gap between the checkpoint of the data to be consumed and the checkpoint of the consumed data. Statistical dimensions: partition, group, and task. |
| Consumption Time Lag of Subscription Partition | SubscribeConsumePartitionLatency | s | Time lag between the consumed data and the data of the source instance. Statistical dimensions: partition, group, and task. |
Metric Name | Parameter | Unit | Description |
LSN Gap Between Subscription Service and Source Database | ProducerLsnLag | MBytes | LSN gap between the offset of parsed logs and that of the latest logs |
Number of Transactions Parsed per Second | ProducerTps | Count/s | Number of transactions parsed from the source instance binlog per second |
Feedback