TencentDB for MySQL provides a dual-node instance with one primary and one secondary nodes and a three-node instance with one primary and two secondary nodes. To allow users to perform timely primary-secondary switch and normal services when a fault occurs in the primary instance, Tencent Cloud Smart Advisor-Chaotic Fault Generator provides users primary-secondary switch capabilities, supports users in manual primary-secondary switch, and helps users in verifying primary-secondary switch reliability, data integrity and overall stability of businesses.
Note:
This fault action requires that your TencentDB for MySQL instance be a multi-node architecture. If this condition is not satisfied, the fault action will not be executed. You can go to TencentDB for MySQL console to check instance architecture type and adjust instance architecture.
Experiment Implementation
Step 1: Experiment Preparation
A three-node MySQL instance cross different availability zones (with the primary nodes in ap-guangzhou-3 and secondary nodes in ap-guangzhou-6 and ap-guangzhou-3)
2. Fill in the basic information of an action group, select MySQL for Object Type, and click Add MySQL Instance.
3. Add Switching primary-secondary roles of instance action.
4. Configure action parameters, Switch Mode is Prefer switching across availability zones.
Note:
The platform provides two fault injection switch modes for primary-secondary switch actions: Prioritize Switch in the Same Availability Zone and Prioritize Cross-Availability Zone switch. Users can select a mode based on specific scenes. The platform recommends the following mode:
1. Primary Node Fault Scene: It is advised that priority be given to switch in the same availability zone. In simulating a primary node fault, select a secondary node in the same availability zone under a normal HA mechanism as the primary node, and the original primary node will become a secondary node.
2. Power outage scene in an availability zone where the primary node is located: An instance is required for cross-availability zone deployment, it is advised that prioritized cross-availability zone switch action be used to simulate the primary node migrated to another availability zone due to primary node availability zone fault.
5. The platform has built-in Tencent Cloud Observability. Monitoring Metrics can be selected. After confirmation, click Submit to create an experiment.
Step 3: Experiment Execution
1. Connect MySQL instances through CVM:
2. In a new chaos engineering experiment, click Execute fault action.
3. Check action execution logs and instance status.
After the fault is completed, the primary node will be in ap-guangzhou-6, and the two secondary nodes will be in ap-guangzhou-3.
When a persistent connection is interrupted, client driver will automatically start reconnection when an inquiry is initiated once again.
