tencent cloud

Elastic MapReduce

Release Notes and Announcements
Release Notes
Announcements
Security Announcements
Product Introduction
Overview
Strengths
Architecture
Features
Use Cases
Constraints and Limits
Technical Support Scope
Product release
Purchase Guide
EMR on CVM Billing Instructions
EMR on TKE Billing Instructions
EMR Serverless HBase Billing Instructions
Getting Started
EMR on CVM Quick Start
EMR on TKE Quick Start
EMR on CVM Operation Guide
Planning Cluster
Administrative rights
Configuring Cluster
Managing Cluster
Managing Service
Monitoring and Alarms
TCInsight
EMR on TKE Operation Guide
Introduction to EMR on TKE
Configuring Cluster
Cluster Management
Service Management
Monitoring and Ops
Application Analysis
EMR Serverless HBase Operation Guide
EMR Serverless HBase Product Introduction
Quotas and Limits
Planning an Instance
Managing an Instance
Monitoring and Alarms
Development Guide
EMR Development Guide
Hadoop Development Guide
Spark Development Guide
Hbase Development Guide
Phoenix on Hbase Development Guide
Hive Development Guide
Presto Development Guide
Sqoop Development Guide
Hue Development Guide
Oozie Development Guide
Flume Development Guide
Kerberos Development Guide
Knox Development Guide
Alluxio Development Guide
Kylin Development Guide
Livy Development Guide
Kyuubi Development Guide
Zeppelin Development Guide
Hudi Development Guide
Superset Development Guide
Impala Development Guide
Druid Development Guide
TensorFlow Development Guide
Kudu Development Guide
Ranger Development Guide
Kafka Development Guide
Iceberg Development Guide
StarRocks Development Guide
Flink Development Guide
JupyterLab Development Guide
MLflow Development Guide
Practical Tutorial
Practice of EMR on CVM Ops
Data Migration
Practical Tutorial on Custom Scaling
API Documentation
History
Introduction
API Category
Cluster Resource Management APIs
Cluster Services APIs
User Management APIs
Data Inquiry APIs
Scaling APIs
Configuration APIs
Other APIs
Serverless HBase APIs
YARN Resource Scheduling APIs
Making API Requests
Data Types
Error Codes
FAQs
EMR on CVM
Service Level Agreement
Contact Us

Custom Scaling Configuration

PDF
Mode fokus
Ukuran font
Terakhir diperbarui: 2026-01-15 15:10:49

Basic Settings

The basic settings section allows you to set the node count range for custom scaling, configure the elastic resource type, and configure whether to allow graceful scale-in for Auto Scaling (AS). It also displays the number of elastic node resources in the current cluster and supports one-click release of elastic instances.
Parameter Settings
Description
Min Node Count
The minimum number of AS task nodes to be retained in the cluster when the automatic scale-in policy is triggered.
Max Node Count
The maximum number of AS task nodes to be retained in the cluster when the automatic scale-out policy is triggered. The cumulative number of nodes scaled out based on single or multiple specifications cannot exceed the max node count.
Release All
Clears all nodes scaled out by auto-scaling with one click. Nodes not scaled out by auto-scaling are not affected.
Release Spot Instances
Clears only the spot instance nodes scaled out by auto-scaling with one click. Non-spot instance resource nodes are not affected.
Release Pay-As-You-Go Instances
Clears pay-as-you-go instance nodes scaled out by auto-scaling with one click. Pay-as-you-go nodes not scaled out by auto-scaling are not affected.
Allow Graceful Scale-in
This feature is disabled by default. The graceful scale-in policy is not enabled in all scale-in rules. When this feature is enabled, if both graceful scale-in and a single rule are enabled, the graceful scale-in policy takes effect.
Resource Type
The HOST resource type supports pay-as-you-go billing and spot instance billing, while the POD resource type only supports pay-as-you-go billing and can only be used to deploy the NodeManager role of Yarn.
Note:
When the resource type is switched, the corresponding scaling specifications and node selection policy are switched and take effect together.

Scaling Specifications Management

Scaling specifications refer to specifying the scale-out node specifications and node selection policy for custom scaling. To maintain a linear change in cluster load, it is recommended to keep the CPU and memory of the scaling specifications consistent.
Node selection policy: Two policies are supported, that is, Pay-as-you-go and Spot instances preferred.
Pay-as-you-go: When a scale-out rule is triggered, only pay-as-you-go nodes are added to supplement computing power.
Spot instances preferred: When a scale-out rule is triggered, spot instances are preferred to supplement computing power. If spot instance resources are insufficient, pay-as-you-go resources are used to make up for the computing power.
Min proportion of pay-as-you-go nodes: The minimum proportion of pay-as-you-go nodes to the scale-out quantity in a single scale-out.
Example:
If 10 nodes are to be added in a single scale-out and the minimum proportion of pay-as-you-go nodes is 20%, when the scale-out rule is triggered, at least 2 pay-as-you-go nodes will be added, and the remaining 8 nodes will be supplemented with spot instances. If spot instance resources are insufficient for 8 nodes, pay-as-you-go nodes will be used to make up the difference.
You can add, delete, change, and query the nodes in the scaling specifications, and adjust the priority of scaling specifications as needed. The rule priority order is from high to low ( 1 > 2 > 3 > 4 > 5).
Note:
When the resource type is preset to POD in the basic settings, the node selection policy only supports pay-as-you-go.

Scaling Rule Management

Scaling rules are business policies used to configure the trigger conditions for scaling actions and change the number of nodes. Two types of scaling policies are supported, that is, load-based scaling and time-based scaling. You can choose the corresponding policy and set scaling rules according to business needs. Also, mixed scaling rules combining time-based and load-based scaling are supported. Rule triggering follows the principle of "first triggered, first executed; if triggered simultaneously, execute according to rule priority order".

Setting Load-based Scaling

When it is impossible to accurately estimate the peaks and troughs of cluster computing, you can configure the load-based scaling policy to ensure that important jobs are completed on time. The load is mainly based on the preset YARN or Trino metric statistical rules, and task nodes are automatically adjusted when the preset conditions are triggered.
Note:
For details on cluster queue load metrics, refer to Corresponding Relationships of Queue Load Metrics.
Click Add Rule. On the "Create Rule" page, select "By load" as the policy type, and set the rule as follows:
Configuration Item
Description
Rule Type
Scale Out / Scale In
Policy Type
By load
Rule Name
The name of the scaling rule. The scaling rule names in the same cluster must be unique (including scale-out and scale-in rules).
Validity
The validity time range during which the load-based scaling rule is triggered. Unlimited is selected by default. Custom time periods are supported for load-based scaling rule configuration.
Load Type
Supports YARN or Trino load metrics. Trino load-based scaling is only supported for clusters deployed with the Trino component in EMR-V2.7.0 and EMR-V3.40 or later versions.
Statistical Rule
Sets single or multiple rules for triggering thresholds simultaneously based on the selected cluster load metrics. Up to 5 statistical rules can be set, and rules can be aggregated based on subqueues.
Rule: Specifies the queue and load metric, and sets the conditional rule for triggering thresholds.
Statistical Period: The selected load metric is triggered once when the trigger threshold is reached within a statistical period according to the selected aggregation dimension (average value, maximum value, and minimum value). Currently, three statistical periods are supported: 300 seconds, 600 seconds, and 900 seconds.
Repeat Count: The number of times that the aggregated load metric reaches the threshold. When the repeat count is reached, the cluster AS action is triggered.
Scale-out / Scale-in Mode
Three options are available, that is, Node, Memory, and Core, and their values must be non-zero integers. When Core or Memory is selected, the number of nodes to be added in case of scale-out is calculated while ensuring maximum computing power; the minimum number of nodes to be released in case of scale-in is calculated while ensuring normal business operation. Also, ensure that the nodes are released in reverse chronological order and at least 1 node is released.
Scale-out Service
By default, the scaling component inherits the cluster-level configuration, and the scale-out nodes belong to the default configuration group for that node type. To adjust the configuration of the scaling component, you can specify configuration settings.
Node Label
By default, resources scaled out without a label are placed in the Default Label. After setting, resources scaled out are placed in the specified Label.
Resource Supplement Retry
When auto-scaling is performed during peak hours, the number of nodes added may fail to reach the target number due to the lack of resources. When you enable the resource supplement retry policy, if the configured scaling specifications resources are sufficient, the system will automatically retry applying for resources until the target number is reached or approached. If insufficient resources often cause auto-scaling to fall short of expectations, you can try enabling this configuration. After you enable it, if a retry is triggered, the auto-scaling time may be extended. Please pay attention to the impact of policy adjustments on business.
Cooldown Period
The interval (cool-down time range: 0 to 43,200 seconds) before the next auto-scaling action is initiated after the current rule is successfully executed.
Graceful Scale-in
After the graceful scale-in mode is enabled, if a scale-in action is triggered while a node is executing tasks, the node will not be released immediately but will wait for the tasks to be completed within a custom time period before scaling in. If the tasks are not completed by the end of the custom time period, scale-in will proceed anyway.

Setting Time-Based Scaling

When the computational load of the cluster exhibits significant peaks and troughs within a certain period, you can configure the time-based scaling policy to ensure that important jobs are completed on time. The time-based scaling policy allows you to add or remove task nodes during a fixed time period every day, week, or month.
Click Add Rule. On the Create Rule page, select By time as the policy type, and set the rule as follows:
Configuration Item
Description
Rule Type
Scale out / Scale in
Policy Type
By time
Rule Name
The name of the scaling rule. The scaling rule names in the same cluster must be unique (including scale-out and scale-in rules).
Execution Type
Once: Triggers a scaling action at a specific time, accurate to the minute.
Recurring: Triggers a scaling action daily, weekly, or monthly at a specific time or time period.
Execution Time: The specific time of executing scaling actions every day.
Validity: The validity range for the repeated execution of a single rule.
Scale-out / Scale-in Mode
Three options are available, that is, Node, Memory, and Core, and their values must be non-zero integers. When Core or Memory is selected, the number of nodes to be added in case of scale-out is calculated while ensuring maximum computing power; the minimum number of nodes to be released in case of scale-in is calculated while ensuring normal business operation. Also, ensure that the nodes are released in reverse chronological order and at least 1 node is released.
Scale-out Service
By default, the scaling component inherits the cluster-level configuration, and the scale-out nodes belong to the default configuration group for that node type. To adjust the configuration of the scaling component, you can specify configuration settings.
Node Label
By default, resources scaled out without a label are placed in the Default Label. After setting, resources scaled out are placed in the specified Label.
Resource Supplement Retry
When auto-scaling is performed during peak hours, the number of nodes added may fail to reach the target number due to the lack of resources. When you enable the resource supplement retry policy, if the configured scaling specifications resources are sufficient, the system will automatically retry applying for resources until the target number is reached or approached. If insufficient resources often cause auto-scaling to fall short of expectations, you can try enabling this configuration. After you enable it, if a retry is triggered, the auto-scaling time may be extended. Please pay attention to the impact of policy adjustments on business.
Retry Time After Expiration
If an AS action cannot be executed at the specified time due to various reasons, setting the retry time after expiration will allow the system to attempt execution periodically within that set time range until the AS conditions are met.
Cooldown Period
The interval (cool-down time range: 0 to 43,200 seconds) before the next auto-scaling action is initiated after the current rule is successfully executed.
Scheduled Termination
Specifies the usage duration for scale-out resources, and when a scale-in rule is triggered, the current batch of nodes will not be affected by the scale-in rule. By default, "Unlimited" is selected. Custom durations are supported. Enter an integer in the range of 1-24 hours.
Use case: This field is used when you need to supplement computing power during fixed time periods and maintain the computing power within a day's range, and other scale-in rules do not affect this batch of resources.
Graceful Scale-in
After the graceful scale-in mode is enabled, if a scale-in action is triggered while a node is executing tasks, the node will not be released immediately but will wait for the tasks to be completed within a custom time period before scaling in. If the tasks are not completed by the end of the custom time period, scale-in will proceed anyway.


Bantuan dan Dukungan

Apakah halaman ini membantu?

masukan