Custom Rate Limiting Rules
Overview
In site operations, issues such as malicious resource occupation, service abuse, and brute-force attacks frequently occur. If overlooked, these problems can lead to degraded service quality, high-cost bills, and even potential leakage of sensitive data. To effectively manage these risks, client access frequency serves as a critical metric. Malicious clients typically access at higher frequencies to rapidly achieve goals like credential cracking, resource occupation, and content scraping. When an appropriate threshold is set to restrict client access frequency, legitimate clients can be effectively distinguished from malicious ones, thereby mitigating risks of resource occupation and abuse.
Note:
1. When crawlers are managed and combated, using only rate limiting policies has limited effectiveness. Please combine the Bot Management feature to formulate a comprehensive crawler management policy.
2. Rate limiting rules are counted separately per edge access server IP address, which can be used to combat HTTP DDoS attacks and ensure origin server availability to some extent. These rules cannot precisely control origin-pulling QPS. For origin-pulling QPS control requirements, see Origin-pull Rate Limit Policy.
Typical Scenarios and Configuration Methods
Rate limiting is commonly used to distinguish legitimate client access from malicious access. By selecting appropriate statistical methods, restriction thresholds, and handling measures, rate limiting can help you mitigate security risks. Rate limiting configurations are categorized into the following types:
Precise Rate Limiting: A user-defined policy for controlling access frequency. It supports combining multiple conditions to match requests and restricts the request rate per source. This is suitable for distinguishing legitimate user access from malicious high-frequency access in most scenarios.
Managed Custom Policy: A policy customized by Tencent security experts that cannot be adjusted in the console. For details, see Managed Custom Rules.
Precise Matching Rules
Scenario 1: Restricting access rate to login APIs to mitigate credential stuffing and brute-force attacks
In scenarios involving credential stuffing and brute-force attacks, attackers frequently attempt to access login API interfaces to obtain or crack information. By restricting the request frequency to login interfaces, we can significantly mitigate attackers' cracking attempts, effectively defending against such attacks and protecting sensitive information from leakage. For example: the site domain
www.example.com provides an external interface /api/UpdateConfig, which allows an access frequency of 100 times/minute. When this frequency limit is exceeded, the IP address will be blocked for 10 minutes. The operational steps are as follows:1. Log in to the Tencent Cloud EdgeOne console, enter Service Overview in the left menu bar, and click the site to be configured under Website Security Acceleration.
2. Click Security Protection > Web Protection. By default, site-level protection policies are applied. Click the Domain-level Protection Policy Tab, then select Target Domain to go to the target domain protection policy configuration page, for example:
www.example.com.3. Locate the Rate Limiting card and click Add Rule under Precise Rate Limiting.
4. Go to the Add Rule page, select the Write interface rate limit rule template, and click Add.
Note:
The configuration fields and sample values in the rule templates are for illustrative purposes only. Please evaluate based on your specific business requirements and normal traffic levels, and adjust the trigger conditions, rate thresholds, and enforcement actions for protection accordingly.
5. Configure the judgment conditions, rate threshold, and enforcement action. Consider the example scenario: configure match fields where Request Method (Method) equals
POST or HEAD; and Request Path (Path) equals /api/UpdateConfig. Adjust the rate threshold to trigger when exceeding 100 times within a counting period of 1 minute. Set the post-trigger action to Block with a duration of 10 minutes.6. After Save and Publish is clicked, the rule will be deployed and take effect.
Example scenario 2: Limit the request rate causing 404 status codes to mitigate random resource scanning
When malicious clients randomly scan site image resources attempting to crawl content, they often trigger origin servers to return
404 errors due to non-existent access paths. By limiting the request frequency that causes origin servers to return 404 status codes, EdgeOne can prevent malicious attackers from large-scale scanning and requesting static resources. This reduces error responses from origin servers, alleviates server load, and enhances the security and stability of static resource sites. For example: for image static resources (.jpg, .jpeg, .webp, .png, .svg) under site domain www.example.com, when resources return 404 errors, if accesses exceed 200 times within 10 seconds, the corresponding client IP address requests will be blocked for 60 seconds. The operational steps are as follows:1. Log in to the Tencent Cloud EdgeOne console, enter Service Overview in the left menu bar, and click the site to be configured under Website Security Acceleration.
2. Click Security Protection > Web Protection. By default, site-level protection policies are applied. Click the Domain-level Protection Policy Tab, then select Target Domain to go to the target domain protection policy configuration page, for example:
www.example.com.3. Locate the Rate Limiting card and click Add Rule under Precise Rate Limiting.
4. Go to the Add Rule page, select the Random Scan rule template, and click Add.
Note:
The configuration fields and sample values in the rule templates are for illustrative purposes only. Please evaluate based on your specific business requirements and normal traffic levels, and adjust the trigger conditions, rate thresholds, and enforcement actions for protection accordingly.
5. Configure the judgment conditions, rate threshold, and enforcement action. Taking the example scenario, configure match fields where HTTP Status Code (supported in Enterprise Edition) equals
404; and Request Path (Path) has file suffix match content as .jpg, .jpeg, .webp, .png, and .svg. Adjust the rate threshold to trigger when the count exceeds 200 times within a counting period of 1 minute. Set the post-trigger action to Block with a duration of 1 minute.
6. After Save and Publish is clicked, the rule will be deployed and take effect.
Example scenario 3: Restrict high-concurrency search engine crawlers from accessing Web sites to mitigate impacts on normal business operations
A Y search engine vendor uses a large-scale distributed crawler architecture with insufficient restrictions on access behavior, resulting in aggressive crawling activities that generate high volumes of traffic in short periods. This may impact normal business operations and consume excessive resources. Therefore, rate limiting is implemented to identify and restrict such crawler access to mitigate its impact. For example: Site
www.example.com experienced disruption to normal business due to high-frequency access by Y search engine crawlers. Through Web Security Analysis, it was observed that Y's distributed crawlers exhibit clustered characteristics in JA3 fingerprints and User-Agents. Thus, a rate limiting rule is configured to block requests sharing the same JA3 fingerprint and User-Agent when the number of requests exceeds 60 within a 30-second statistical window. Blocking persists for 10 minutes. The operational steps are as follows:1. Log in to the Tencent Cloud EdgeOne console, enter Service Overview in the left menu bar, and click the site to be configured under Website Security Acceleration.
2. Click Security Protection > Web Protection. By default, site-level protection policies are applied. Click the Domain-level Protection Policy Tab, then select Target Domain to go to the target domain protection policy configuration page, for example:
www.example.com.3. Locate the Rate Limiting card and click Add Rule under Precise Rate Limiting.
4. Go to the Add Rule page, select "Create Blank Rule", fill in the rule name, and click Add.
5. Configure the judgment conditions, rate threshold, and enforcement action. By taking the example scenario, configure match fields where Application-layer Protocol equals
HTTPS, and clients sharing identical characteristics in JA3 Fingerprint and HTTP Header User-Agent. Set the rate threshold to trigger when exceeding 60 times within a counting period of 30 seconds. Set the post-trigger action to Block with a duration of 10 minutes.
6. After Save and Publish is clicked, the rule will be deployed and take effect.
Reference
When creating a rate limiting rule, you need to configure the rule matching objects, trigger conditions, and enforcement actions. The configuration items are described as follows:
Note:
If your current rate rule needs to match based on a known HTTP header value, configure Match Condition > Specified Match Field as Specified HTTP Header Parameter Value.
If your current rate rule needs to match based on a category of HTTP headers that may share the same value, configure Rate Threshold > Request Characteristics as HTTP Header with Specified NameHeader for matching.
Match Object
Set a combination of match conditions based on request source, header characteristics, response status codes, and so on.1. Rate limiting rules only control traffic that meets these conditions. For details about match conditions and support levels across different plans, see Match Conditions.
Trigger Method
Note:
1. When the request rate does not reach the threshold, no action will be taken and the requests will not be logged.
2. Specific configuration options and configurable ranges vary depending on the subscription plan. For details, see Edition Comparison.
3. In the counting cycle options, the 1 second option only supports the Client IP and Client IP (preferentially matches XFF header) request characteristics.
The rule will count requests based on the statistical rules configured in the trigger conditions. When the accumulated request count exceeds the threshold within the counting cycle, the rule triggers and executes the corresponding restrictive action2. Statistics are based on the counting cycle and counting method, where requests are counted per distinct feature value (such as client IP) under specified feature dimensions1. You can configure the following parameters for trigger conditions:
Counting cycle: The length of the rolling time window used for counting. Supported range is from 1 second to 1 hour.
Counting method: The approach to distinguish request sources. Rate limiting restricts the request rate per request source. For details, see Counting Dimensions.
Rate threshold: The number of allowed requests per source (such as Client IP address) within a counting cycle.
Duration of Triggered State: The period during which requests from the source matching the conditions continue to be restricted3 after the rule is triggered. Supported range is from 1 second to 30 days.
Request Characteristics
The system supports statistics based on one or more request characteristics. When requests within a statistical dimension meet the rate threshold set in the trigger conditions, the rate limiting rule is triggered. You can specify the following counting dimensions1:
Client IP address: Requests from the same source IP address will be counted in the same counter. When the threshold is exceeded, the rule's action will be triggered.
Client IP (preferentially matches XFF header): Requests from the same client IP address will be counted in the same counter. When the threshold is exceeded, the rule's action will be triggered. When the X-Forwarded-For header is present and contains valid IP addresses, the first IP address in the X-Forwarded-For header will be prioritized for counting.
Specified Cookie Name: Extract the value of the specified Cookie from the request header. Requests with the same Cookie value are counted in the same counter. When the threshold is exceeded, the rule's action will be triggered.
For example: When a website uses a Cookie named
user-session to track access sessions, you can configure the Cookie value with the specified name user-session as the counting dimension to track request rates per session. When the request rate in a single session exceeds the threshold, the action configured in the rule will be triggered.Specified HTTP Header Name: Extract the value of the specified header from the request header. Requests with the same header value are counted in the same counter. When the threshold is exceeded, the rule's action will be triggered. For example: You can specify the Origin header to limit the access frequency from each external domain. When the access frequency from a particular external domain exceeds the threshold, the action configured in the rule will be triggered.
Specified URL Query Parameter: Extract the value of the specified parameter from the request URL query. Requests with the same query parameter value are counted in the same counter. When the threshold is exceeded, the rule's action will be triggered.
For example: When a website uses a query parameter named
user-session to track access sessions, you can configure the query parameter with the specified name user-session as the counting dimension to track request rates per session. When the request rate in a single session exceeds the threshold, the action configured in the rule will be triggered.JA3 Fingerprint of Request4: Calculate the JA3 fingerprint for each request. Requests with identical JA3 fingerprints are counted collectively. When the threshold is exceeded, the rule's configured action will be triggered. Each request corresponds to a unique JA3 fingerprint value, and no key-value model is involved, so no specific parameters need to be entered. Considering the characteristics of JA3, it is recommended to configure it alongside the User-Agent header as a counting dimension to effectively distinguish between clients.
JA4 Fingerprint of Request: Calculate the JA4 fingerprint for each request. Requests with identical JA4 fingerprints are counted collectively. When the threshold is exceeded, the rule's configured action will be triggered. Each request corresponds to a unique JA4 fingerprint value, and no key-value model is involved, so no specific parameters need to be entered.
Access Path of Request: Extract the access path of the request (URL Path, excluding URL query parameters). Requests with the same access path are counted in the same counter. When the threshold is exceeded, the rule's action will be triggered.
Note:
Note 1
: The supported matching conditions, statistical dimensions, and action options may vary depending on your subscribed plan. For details, see Package Options Comparison.Note 2:
If multiple rate limiting rules exist, a single request may match multiple rules simultaneously. Each rule will independently determine whether to trigger based on its own statistical method. When a request triggers one rule and gets blocked, other rules will not be triggered. When multiple rules are triggered simultaneously, they will execute according to their priority order, with rules having smaller priority values taking precedence. For details, see Web Protection Request Processing Order.Note 3
: When a rule is triggered, it will only take effect on requests that match the current rule.Note 4
: JA3 fingerprint is identification information formed based on the client's TLS information, which can effectively distinguish requests from different Bot networks. When a request is initiated via non-SSL HTTP protocol, its JA3 fingerprint will be empty. To use JA3 fingerprint, ensure that Bot Management feature is enabled for your current domain.Note 5: To count the number of requests with identical characteristics by combining multiple statistical dimensions, you need to subscribe to the EdgeOne Enterprise Edition plan.
Action
When requests exceed the limit threshold, corresponding restriction actions will be applied. Supported actions include Block, Monitor, JSChallenge, Redirect, and ReturnCustomPage1. For detailed descriptions of enforcement methods, see Disposal Methods.
Help and Support
Was this page helpful?
You can also Contact sales or Submit a Ticket for help.
Help us improve! Rate your documentation experience in 5 mins.
Feedback