Tencent Cloud

Recent Pages

High CPU Utilization

Last updated: 2023-05-23 10:30:16

Increasing CPU utilization will affect the throughput of the entire instance cluster, and may even cause application blocking and timeout interruptions. When the average CPU utilization exceeds 60% or the average peak CPU utilization stays higher than 90% for more than 5 minutes, you need to check the specific cause in time and solve the problem quickly to ensure business stability and availability.
Symptom
Symptom 1: You received an alarm about high CPU utilization.
Symptom 2: The value of the CPU utilization metric was high.
Symptom 3: The overall throughput became lower and the response slower.
Troubleshooting
No.
Possible Causes
Cause Analysis
Troubleshooting Methods
Solution
1
Non-persistent connections are established frequently.
A large number of resources of the instance are consumed in processing frequent non-persistent connections, resulting in high CPU utilization and a lot of connections. However, the QPS (cluster accesses per second) did not meet expectations.
With DBbrain's Performance Optimization feature, you can analyze the real-time session statistics view and data of the database instance to check whether there is a sudden increase in the number of connections. For directions, see Real-Time Session.
Change the non-persistent connection to a persistent connection, for example, for example, by using the JedisPool connection pool. For specific code samples, see Jedis Client.
2
There are commands with high time complexity, such as sort, sunion, and zunionstore.
As Redis executes commands in a single thread, the execution of complex commands may block other commands. The higher the time complexity of the command, the more resources will be consumed during execution and the CPU utilization will increase.
Executing high-complexity commands is usually time-consuming and will generate slow logs accordingly. You can use DBbrain's Performance Optimization feature to perform Slow Log Analysis, and check whether there are high-complexity commands in the slow log information list. For directions, see Slow Log Analysis.
Avoid using complex commands to get large amounts of data. Try to just use them on a limited amount of data instead, so Redis can process it quickly and return the results.
3
The read and write load is too high due to big keys or a large number of accesses to hot keys.
Hot keys refer to those key values that are accessed very frequently over a period of time, such as trending news, popular live streaming, and flash sales. In these cases, a large amount of access traffic is concentrated in one instance, reaching the processing capacity of a single instance and causing an increase in CPU utilization.
With DBbrain's Performance Optimization feature, you can perform Hot Key Analysis to quickly find hot keys with high access frequency. For directions, see Hot Key Analysis.
You can split hot keys of complex data structures into several new keys and distribute them across Redis nodes to reduce the pressure. For example, if a two-level hash hot key has a lot of hash elements, you can split it.
4
﻿
A big key is one that has big value and takes up a large space. When read or deletion operations related to big keys are performed, bandwidth and CPU will be seriously occupied.
With DBbrain's Performance Optimization feature, you can perform Big Key Analysis and monitor and analyze the memory usage of big keys in the database. For directions, see Memory Analysis.
If the value of a big key is too large, you can split the big key into multiple key-values so that multiple Redis nodes will share the pressure. If there are too many keys, you can store them in a hash structure.
5
﻿
The read load is too high and the resource limit has been reached.
With DBbrain's Performance Optimization feature, you can check the Performance Trends view of the database instance, analyze QPS and read/write request metrics, and check whether the high CPU utilization is caused by excessive read load or write load. For directions, see Performance Trends.
If the read load is too high, you can share the read load by increasing the number of replicas as  instructed in Changing Instance Specification. You need to enable read-only replicas and transfer the read requests of the current instance to the read-only replica node, so as to realize the elastic expansion of the read capability and improve the read and write performance. For directions, see Enabling/Disabling Read/Write Separation.
6
﻿
The write load is too large, and the memory specification is insufficient.
﻿
If the write load is too high, you can share the write load by increasing the number of shards as instructed in Changing Instance Specification. If the instance is on a standard architecture, you need to upgrade the standard architecture to a cluster architecture to improve CPU processing capabilities. For directions, see Upgrading Instance Architecture. You need to check compatibility before upgrading to the cluster architecture. For more information, see Check on Migration from Standard Architecture to Cluster Architecture.
7
A large number of keys expire at the same time.

The key expiration time is set at the same time point. When the expiration time point is reached, Redis will take up a lot of CPU resources to process the elimination thread of those expired keys, causing a momentary lagging.

On the System Monitoring page in the console, you can check the Expired Keys metric to confirm whether a large number of keys expire at a certain point in time. For directions, see Update Notes of Monitoring at Five-Second Granularity.

You can disperse the expiration time of those keys in the business logic to avoid them all expiring at the same time.
8
Databases are frequently switched by executing the select command.
Frequent database switch leads to excessive resource overhead.

With DBbrain’s Latency Analysis feature, you can check the monitoring data of the select command in Command Word Analysis, and confirm whether there are many select requests. For directions, see Command Word Analysis.
For the storage of different businesses, if some of them require frequent database switch, we recommend that you store them separately.
For the storage of the same businesses, if the key names do not conflict, you can consider storing them in the same database to reduce the number of select request operations.
﻿
﻿
﻿
﻿
﻿
﻿

Contact Us

Contact our sales team or business advisors to help your business.

Technical Support

Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

7x24 Phone Support

No.	Possible Causes	Cause Analysis	Troubleshooting Methods	Solution
1	Non-persistent connections are established frequently.	A large number of resources of the instance are consumed in processing frequent non-persistent connections, resulting in high CPU utilization and a lot of connections. However, the QPS (cluster accesses per second) did not meet expectations.	With DBbrain's Performance Optimization feature, you can analyze the real-time session statistics view and data of the database instance to check whether there is a sudden increase in the number of connections. For directions, see Real-Time Session.	Change the non-persistent connection to a persistent connection, for example, for example, by using the JedisPool connection pool. For specific code samples, see Jedis Client.
2	There are commands with high time complexity, such as `sort`, `sunion`, and `zunionstore`.	As Redis executes commands in a single thread, the execution of complex commands may block other commands. The higher the time complexity of the command, the more resources will be consumed during execution and the CPU utilization will increase.	Executing high-complexity commands is usually time-consuming and will generate slow logs accordingly. You can use DBbrain's Performance Optimization feature to perform Slow Log Analysis, and check whether there are high-complexity commands in the slow log information list. For directions, see Slow Log Analysis.	Avoid using complex commands to get large amounts of data. Try to just use them on a limited amount of data instead, so Redis can process it quickly and return the results.
3	The read and write load is too high due to big keys or a large number of accesses to hot keys.	Hot keys refer to those key values that are accessed very frequently over a period of time, such as trending news, popular live streaming, and flash sales. In these cases, a large amount of access traffic is concentrated in one instance, reaching the processing capacity of a single instance and causing an increase in CPU utilization.	With DBbrain's Performance Optimization feature, you can perform Hot Key Analysis to quickly find hot keys with high access frequency. For directions, see Hot Key Analysis.	You can split hot keys of complex data structures into several new keys and distribute them across Redis nodes to reduce the pressure. For example, if a two-level hash hot key has a lot of hash elements, you can split it.
4			A big key is one that has big value and takes up a large space. When read or deletion operations related to big keys are performed, bandwidth and CPU will be seriously occupied.	With DBbrain's Performance Optimization feature, you can perform Big Key Analysis and monitor and analyze the memory usage of big keys in the database. For directions, see Memory Analysis.	If the value of a big key is too large, you can split the big key into multiple key-values so that multiple Redis nodes will share the pressure. If there are too many keys, you can store them in a hash structure.
5			The read load is too high and the resource limit has been reached.	With DBbrain's Performance Optimization feature, you can check the Performance Trends view of the database instance, analyze QPS and read/write request metrics, and check whether the high CPU utilization is caused by excessive read load or write load. For directions, see Performance Trends.	If the read load is too high, you can share the read load by increasing the number of replicas as instructed in Changing Instance Specification. You need to enable read-only replicas and transfer the read requests of the current instance to the read-only replica node, so as to realize the elastic expansion of the read capability and improve the read and write performance. For directions, see Enabling/Disabling Read/Write Separation.
6			The write load is too large, and the memory specification is insufficient.			If the write load is too high, you can share the write load by increasing the number of shards as instructed in Changing Instance Specification. If the instance is on a standard architecture, you need to upgrade the standard architecture to a cluster architecture to improve CPU processing capabilities. For directions, see Upgrading Instance Architecture. You need to check compatibility before upgrading to the cluster architecture. For more information, see Check on Migration from Standard Architecture to Cluster Architecture.
7	A large number of keys expire at the same time.	The key expiration time is set at the same time point. When the expiration time point is reached, Redis will take up a lot of CPU resources to process the elimination thread of those expired keys, causing a momentary lagging.	On the System Monitoring page in the console, you can check the Expired Keys metric to confirm whether a large number of keys expire at a certain point in time. For directions, see Update Notes of Monitoring at Five-Second Granularity.	You can disperse the expiration time of those keys in the business logic to avoid them all expiring at the same time.
8	Databases are frequently switched by executing the `select` command.	Frequent database switch leads to excessive resource overhead.	With DBbrain’s Latency Analysis feature, you can check the monitoring data of the `select` command in Command Word Analysis, and confirm whether there are many `select` requests. For directions, see Command Word Analysis.	For the storage of different businesses, if some of them require frequent database switch, we recommend that you store them separately. For the storage of the same businesses, if the key names do not conflict, you can consider storing them in the same database to reduce the number of `select` request operations.

tencent cloud

Recent Pages

High CPU Utilization

Symptom

Troubleshooting

Was this page helpful?

Was this page helpful?

tencent cloud

Sign Up

Log in

Recent Pages

High CPU Utilization

Symptom

Troubleshooting

Was this page helpful?

Was this page helpful?