TI-ONE resource groups support "queuing policy" and "priority scheduling policy" when scheduling training tasks and online services.
Queuing Policy: When resources are insufficient, the default queuing policy is first-in-first-out based on the submission time of tasks/services. Users can also configure a traversal policy in the resource group to prioritize scheduling tasks/services that meet the resource requirements in the queue. For details, see Queuing Policy Configuration Instructions.
Priority Scheduling Policy: Supports setting the priority (P0 is the highest and P9 is the lowest) of tasks/services based on tags. High-priority tasks/services will preempt low-priority tasks by default. After being preempted, low-priority tasks will re-enter the queue. For detailed usage instructions, see Priority Scheduling Explanation.
Remark: The feature of setting task priority through tags is only supported for the "task-based modeling" and "online service" modules.
Principle of priority for services: High-priority online services can preempt resources of low-priority task-based modeling, but high-priority task-based modeling will not preempt resources of low-priority online services.
Notes:
CPU tasks are evenly distributed to different nodes of the resource group by default according to the Cloud Load Balancer (CLB) policy.
GPU tasks are preferentially scheduled to the same node in a way that minimizes fragments by default.
Tasks of different card types will not block each other. For example, tasks of a T4 GPU card will not affect tasks of an A100 GPU card.
Queueing Policy Configuration Instructions
DefaultQueue indicates the default queuing policy for training tasks within a resource group and is only supported for task-based modeling.
Among them, the default value is 0. Setting it to 0 means strict first-in-first-out: Regardless of the amount of idle resources currently available, the earliest submitted task in the waiting queue is selected. If sufficient resources are available, the task is scheduled for execution; otherwise, it needs to wait.
Set to 1 to represent first-in-first-out with maximum resource utilization: According to the current resource availability, find the first task in the queue that meets the current resource requirements in chronological order and schedule it for execution.
{
"Version":"1.0",
"ResourceRule":{
"DefaultPriority":0,
"DefaultQueue":0,
"Preempted":1
},
"TaskRules":[]
}
Parameter Description
DefaultPriority: Default priority. The priority ranges from 0 to 9. P0 is the highest priority. The default value is 0.
DefaultQueue: Default queuing policy. Available values: 0, 1, and 2. Default value: 0.
0 means strict first-in-first-out: The earliest submitted task in the waiting queue is selected regardless of idle resources currently available. If sufficient resources are available, the task is scheduled for execution; otherwise, it needs to wait.
1 indicates a traversal policy within the priority range, which means that tasks with the same priority do not necessarily have to be dequeued strictly in the order of task submission time and can queue jump based on idle resource status;
2 indicates a traversal policy for all queues, which means that the entire queuing queue follows the traversal policy and a task can be dequeued as long as resources are satisfied.
Preempted: Default priority preemption policy. Available values: 0 and 1. Default value: 1.
0 means high-priority tasks in the queue will not preempt low-priority tasks that are already running;
1 means high-priority tasks in the queue will preempt low-priority tasks that are already running.
Priority Scheduling Explanation
Policy Syntax Example
At the resource group dimension, a description file for configuring the scheduling policy is supported. Among them, DefaultPriority indicates the default priority. In TaskRules, you can set the priority by task tag following the example below. In this example, ValueType defaults to Tag, and can be supplemented based on actual needs subsequently.
The following is an example statement for the scenario where the default priority of training tasks under this resource group is P0. If the tags of a task comply with Task Type - Speech Task, the priority of this task is P1. Note: The function of setting task priorities through tags is only supported for task-based modeling and online services.
Low-priority tasks may be preempted by high-priority tasks or services, but low-priority services will only be preempted by high-priority services. The preempted tasks/services will enter the queue and continue waiting.
Principle of priority for services: High-priority online services can preempt resources of low-priority task-based modeling, but high-priority task-based modeling will not preempt resources of low-priority online services.
Note:
The #comment part in the following policy statement does not meet JSON grammar and is only used for document display. The comment needs to be cleared when the policy statement is pasted to the platform.
// Task scheduling policy, which is a rule matching list {
// Each rule is a matching relationship between attributes and scheduling."AttrRules":[{"ValueType":"Tag","Key":"Task type","Value":"Speech task"}],"ScheduleRule":{"Priority":1// Set priority. Priority range: 0-9}}]}
Specific Operation Process (Taking Task-Based Modeling As an Example)
1. Log in to the Tag console, and select the Tag list.
Click Create Tag. Enter Task Type for the tag key, and Speech Task for the tag value.
2. On the TI-ONE resource group management page, select a resource group. Click Edit, and enter the above policy description statement in the Scheduling Policy column.
3. When creating a new training task, tag this task with Task Type > Voice Task during task creation if you need to set this task to low priority. This task will be a low-priority task and be preempted during scheduling, and will automatically enter the queue.
