Polaris (North Star) supports visualization of circuit breaking rules management and supports setting circuit breaking rules for three isolation levels: service, instance, and API. On top of common circuit breakers, Polaris additionally supports active fault detection.
Circuit Breaking Principle
Definition
Service circuit breaking definition: When the downstream service is unavailable or responds too slowly for some reason, the upstream service will not continue to call the target service and return directly in order to ensure the availability of its overall service. When the downstream service is restored, the upstream service will resume calling.
Circuit Breaker Status
Service circuit breaking involves a key concept: the circuit breaker.
1. With the active detection capability disabled, the state transition of the circuit breaker is as follows:
1.1 Initially in the closed state, once errors or slow responses reach a certain threshold, it will switch to the open state, at which point it will no longer call the downstream target service.
1.2 After waiting for a period of time, it will turn into thehalf open state and attempt to release some requests to downstream services.
1.3 Once a successful response is detected, it will return to theclosed state, i.e. service recovery; otherwise, it will return to the open state.
Where the circuit breaker changes from close to open state, the following two conditions shall be met at the same time:
Prerequisites: there are at least a certain number of requests within the sliding time window (i.e. minimum request number)
Reach threshold: The false request rate or slow request rate measured in the sliding time window reaches a certain threshold.
2. With the active detection capability enabled, the state transition of the circuit breaker is as follows:
While processing business requests, active detection requests are added. Even in the absence of business requests, the operational status can be accurately monitored in a timely manner.
Isolation Level and Scenario
Polaris supports circuit breaking rules at three isolation levels: service, instance, and API.
Isolation Level
Request Statistics Range
Circuit Breaking Object
Applicable scenario
Service
All APIs for all instances of the downstream target service
Service
All instances can be circuit broken when the downstream service is an unimportant business.
Instance
All APIs for a single instance of the downstream target service
Instance that meets a circuit breaking trigger condition
When the downstream service is relatively important, only abnormal instances will be circuit broken to avoid a service avalanche after all instances are circuit broken.
API
Specified API for all instances of the downstream target service
A single API that meets a circuit breaking trigger condition
Different APIs of the downstream service have different degrees of importance, so different circuit breaking policies need to be set for the APIs.
Usage Instructions
To implement service circuit breaking, two steps are required:
Set circuit breaking rules and active detection rules on the console.
2. In the left sidebar, select Polaris (North Star), and then select the target engine instance on the instance details page.
3. Click Circuit Breaking and Degradation in the left sidebar to go to the circuit breaking rules display page, and click New Circuit Breaking Rule.
Rule name: The rule name must be globally unique and cannot exceed 64 characters.
Description: The description of the rule, used to provide additional information about the rule.
Matching conditions are primarily used to determine the scope of application for circuit breaking rules. Clients use these conditions to filter the applicable circuit breaking rules for local invocations.
Calling Service: Configure the service name and namespace of the calling party. (Optional.) If not configured, it takes effect for all callers by default.
Called Service: Configure the service name and namespace of the called party. (Optional.) If not configured, it takes effect for all called services by default.
Called interface: Configure the called interface name, indicating that the circuit breaking rule takes effect only for requests calling this specific interface. (Optional.) The interface name supports multiple matching conditions.
Error judgment criteria: You can configure multiple criteria. Requests that meet any of these criteria will be flagged as erroneous. Two types of judgment methods are supported: return code and latency.
Circuit breaking triggering conditions: You can configure multiple triggering conditions. If any condition is met, circuit breaking is triggered, and the resource enters the circuit breaking state. Two types of triggering conditions are supported: consecutive error count and error rate.
Consecutive error count: The number of consecutive errors in requests to the service/interface. Circuit breaking is triggered when the threshold is reached.
Error rate: The error rate of requests to the service/interface within a statistical period. Circuit breaking is triggered when the threshold is reached. To avoid amplification effects under low traffic, you can configure a minimum request threshold for error rate statistics. Circuit breaking judgment is performed only when the request count exceeds this threshold.
Circuit breaking recovery: After circuit breaking is triggered for a resource, requests to the resource are blocked for a period of time. The system then attempts to recover the resource. The resource returns to normal after a configured number of consecutive successful requests is met. You can configure the circuit breaking duration (in seconds) and the number of consecutive successful requests required for recovery.
You can choose to enable active probing. After enabling, the calling party initiates probes to the called party based on the probing rules associated with the called service/interface. The probe results are combined with business invocations and serve as one of the bases for circuit breaking triggering or recovery. After enabling active probing, you need to configure active probing rules.
2. In the left sidebar, click Service Governance Center > Traffic Management to go to the Traffic Management page.
3. At the top of the page, select the Circuit Breaking and Degradation tab and then the Active Probing tab.
4. Click Create New Active Probing Rule.
Rule name: Required. The rule name must be globally unique.
Description: Optional. The description of the rule, used to provide additional information about the rule.
Namespace: Required. The namespace of the called service.
Service name: Required. The name of the called service.
API Name: Optional. The name of the called API, which supports multiple matching conditions.
Period: Optional. The probing interval, indicating how often probing is executed, default 30 seconds.
Timeout: Optional. The maximum timeout period for probing. If no response is received within this period, the timeout will be used as the basis for circuit breaking. Default: 60 seconds.
Port: Optional. The probing port, which defaults to the service instance port.
Protocol: The protocol used for probing affects the subsequent configuration. Three protocols are supported: HTTP, TCP, and UDP.
HTTP protocol configuration:
Method: HTTP method, default Get.
Url: The probe Url, default /.
Headers: The headers required for sending probe packets, default empty.
TCP protocol configuration: TCP uses tcp connect for probing by default. Users can also configure packet-based probing methods.
Send: The TCP binary packet to be sent, configured as a hexadecimal string starting with 0x, for example: 0x12ab.
Receive: The TCP binary packet to be received. Multiple packets can be configured. If not configured, the response is not validated.
UDP protocol configuration: UDP does not support connection detection and can only be probed using packet-based methods.
Send: The UDP binary packet to be sent, configured as a hexadecimal string starting with 0x, for example: 0x12ab.
Receive: The UDP binary packet to be received. Multiple packets can be configured.
Step 3: Integrate Circuit Breaking Capability on the Client Side
Refer to the chapters on Circuit Breaking and Degradation in the development documentation according to your access method. For example, if you use Spring Cloud for access, see the document Spring Cloud Application Development > Service Circuit Breaking.
