This document introduces the technical architecture of the global database (GD).
Technical Architecture Diagram
The primary cluster and secondary cluster are allowed to be deployed in different regions.
Data reading follows the principle of proximity, prioritizing data reading from the local region.
When data is written, the primary cluster will write it directly to the local storage, while the secondary cluster will forward the write requests to write data to the primary cluster through a database proxy.
Challenge Analysis of Features and Optimization
In the architecture of the GD, the primary and secondary clusters across different regions employ the Redo Log mechanism to achieve data synchronous replication, ensuring strong consistency and high availability. To guarantee that the data latency for the secondary cluster is strictly controlled within 2 seconds, two core capabilities should be possessed:
Ultra-high log transmission speed: By using high-performance inter-region data transmission channels, the massive Redo Logs can be quickly synchronized to cross-region clusters.
Bandwidth optimization in high-latency networks: In long-distance, high-latency cross-region network environments, the effective throughput can be improved to avoid network congestion that impacts synchronization efficiency.
Technical scheme
Parallel out-of-order log transmission. The primary cluster divides logs into three Worker threads for parallel out-of-order sending, using multiple network connections to improve the log transmission throughput and optimize network transmission bandwidth bottlenecks. The secondary cluster receives the logs for playback to generate data.
Optimization effects
The latency is reduced from 1 minute to less than 2 seconds.