Intelligent Highlights Tutorial

Last updated:2026-03-10 17:03:31

Intelligent Highlights Tutorial

Last updated: 2026-03-10 17:03:31

Overview
The Intelligent Highlights feature uses intelligent algorithms to automatically capture and generate video highlights, providing users with quick review and sharing to enhance post-production efficiency.
﻿
﻿
﻿
Highlight Scenarios and Billing
Media Processing Service (MPS) Intelligent Highlights supports the following editions, which are suitable for different video scenarios. For detailed pricing, see Pay-As-You-Go.
Processing Type
Intelligent Highlights Edition
Description
Supported Video Scenarios
Billing
Process offline files
Intelligent Highlights - Large Model Edition
This edition leverages large model-based video understanding to automatically analyze video content and extract key scenes and highlight moments. Video understanding prompts are customizable.
Custom scenarios
Panoramic camera and VLOG videos
Short dramas, films, and TV series
Charged under "Intelligent Highlights - Large Model Edition".
﻿
Intelligent Highlights - Advanced Edition
Advanced algorithms are used in this edition, primarily tailored for sports events and esports videos.
Football matches
Basketball matches
Esports videos
General scenarios
Charged under "Intelligent Highlights - Advanced Edition".
﻿
Intelligent Highlights - Basic Edition
As of December 2025, this edition has been decommissioned and is now only available to existing customers.
﻿
Charged under "Intelligent Highlights - Basic Edition".
Process live streams
Intelligent Highlights - Live Stream Processing
Built on large model-based video understanding, this edition provides targeted model optimizations for live streaming scenarios.
Broadcast and new media live streaming
Online education live streaming
Live shopping
Financial live streaming
Football/basketball match live streaming
Charged under "Intelligent Highlights - Large Model Edition".
Demo
1. Access the Experience Center to navigate to the Intelligent Highlights page. On the right side, select either an offline video file or a live stream, specify the video content category, and click One-Click Processing.
2. Once the processing is complete, you can view the results.
Note:
The function of the MPS Demo is relatively simple, only for experiencing the basic effect, please use the API access to test the complete effect.
﻿
Pre-integration Prerequisites
Before integrating Intelligent Highlights, you need to complete the following prerequisites to use MPS: register and log in to a Tencent Cloud account, activate MPS, and grant permissions to the service role.
For detailed guidance, see Getting Started. For account authorization issues, see Account Authorization.
Initiating an Offline Video File Processing Task
Using Preset Template No. 26 for Intelligent Analysis
To use Intelligent Highlights, you need to initiate a task using preset template No. 26 under "Intelligent Analysis" in MPS. Custom templates are not supported for highlight parameter adjustment. To specify an Intelligent Highlights edition or video scenario, you need to pass extended parameters when initiating a task. For details, see Highlight Scenarios & Examples of Extended Parameters below.
Specifying Offline Highlight Scenarios & Examples of Extended Parameters
Currently, only preset template No. 26 can be used to initiate Intelligent Highlights tasks. Custom templates are not supported for highlight parameter adjustment. To specify an Intelligent Highlights edition or video scenario, you need to pass extended parameters when initiating a task. The following are common highlight scenarios and corresponding examples of extended parameters.
Note:
This section describes the offline file processing scenarios supported by Intelligent Highlights, along with corresponding examples of extended parameters. For instructions on how to initiate a task and pass extended parameters, see the description about initiating a processing task below. For detailed field definitions of extended parameters, see the appendix below.
The example parameters may not produce optimal highlights. It is recommended to fine-tune them based on the video characteristics. For professional support, please contact us.
Intelligent Highlights - Large Model Edition
Custom Scenarios
multimodal_prompt accepts your custom prompts. Example extended parameters:
//Example prompt. Customize as needed. For detailed field definitions, see the appendix below.
{"hht":{"top_clip":5,"force_cls":10020,"prompts":{"multimodal_prompt":"Skiing scenario, output character highlights"},"scenario":"Skiing","model_segment_limit":[3,6]}}
﻿
Panoramic Camera and VLOG Videos
Built on large model-based video understanding and fine-tuned for various scenarios including VLOG videos, sports videos, landscape videos, and drone panoramic videos, this edition accurately identifies and generates high-quality highlights from captured footage. Example extended parameters:
//2D video highlights. For detailed field definitions, see the appendix below.
{"hht":{"top_clip":5,"force_cls":10020,"model_segment_limit":[3,6]}}
﻿
//Panoramic video highlights. For detailed field definitions, see the appendix below.
{"hht":{"top_clip":5,"force_cls":10020,"model_segment_limit":[3,6],"use_panorama_direct":1,"panorama_video":1}}
﻿
Short Dramas, Films, and TV Series
Built on large model-based video understanding and fine-tuned for short dramas, films, and TV series, this edition automatically extracts highlights such as character entrances and BGM moments to enhance post-production efficiency.
{"hht":{"force_cls":"10010","merge_type":0, "need_vad":1, "top_clip":100, "res_save_type":1,"scenario":"TV series highlights"}}
﻿
Intelligent Highlights - Advanced Edition
Football Matches
Based on video content understanding, this edition automatically identifies and extracts key events in football match videos, including shots, goals, penalties, red cards, yellow cards, and replays.
{"hht":{"force_cls":"4001","merge_type":0, "need_vad":1, "top_clip":100, "res_save_type":1}}
﻿
Basketball Matches
{"hht":{"force_cls":"4002","merge_type":0, "need_vad":1, "top_clip":100, "res_save_type":1}}
Method 1: Initiating a Task via an API
1. Quick Verification with API Explorer
Go to the MPS API Explorer online debugging page and select the ProcessMedia API from the API list on the left. Refer to the figure below to configure parameters such as input path, output path, and template ID. In the AiAnalysisTask configuration, set Definition to 26 (preset Intelligent Highlights template) to initiate an Intelligent Highlights task. ExtendedParameter is an Intelligent Highlights extended parameter, used to specify different highlight scenarios and enable specific capabilities. For details, see Highlight Scenarios & Examples of Extended Parameters.
Note:
API Explorer automatically escapes JSON strings. You can directly enter the JSON data in the ExtendedParameter field without manually escaping it. However, if you call the API directly, you need to escape the JSON string.
﻿
﻿
﻿
﻿2. Initiating via API
The above explained how to use the API Explorer to call and debug the interface online. You can also directly initiate a POST request to Tencent Cloud. API request domain: mps.tencentcloudapi.com. While initiating a POST request, the definition is the preset intelligent highlight template ID (26). Below is a reference example for the  ProcessMedia request:
Note:
When directly calling the API, you need to escape the JSON string when inputting the ExtendedParameter parameter.
{
    "InputInfo": { //Input file path.
        "Type": "URL",
        "UrlInputInfo": {
            "Url": "https://mg-aidata-1258344699.cos-internal.ap-guangzhou.tencentcos.cn/test/hht_test/MyStoryForYouEP39.mp4"
        }
    },
    "OutputStorage": {//Output file storage path.
        "Type": "COS",
        "CosOutputStorage": {
            "Bucket": "mg-aidata-1258344699",
            "Region": "ap-guangzhou"
        }
    },
    "OutputDir": "/test_data/",
    "AiAnalysisTask": {
        "Definition": 26, //26 indicates using the Intelligent Highlights feature.
        "ExtendedParameter": "{\\"hht\\":{\\"top_clip\\":5,\\"force_cls\\":10020,\\"prompts\\":{\\"multimodal_prompt\\":\\"Skiing scenario, output character highlights\\"},\\"scenario\\":\\"Skiing\\",\\"model_segment_limit\\":[3,6]}}" //Extended parameters for specifying highlight scenarios and enabling specific capabilities. See relevant documentation for parameter details.
    }
}
Method 2: Console Task Initiation (Zero-Code Automatic Generation)
Note:
Initiating a task from the console requires a preset template (preset intelligent highlights parameters). Due to some special parameters in the highlights that cannot be configured in the template, it may affect the effect of the Intelligent Highlights. Therefore, it is recommended to use the API integration.
1. Go to the MPS console and click Create Task > Create Offline File Processing Task.
﻿
2. First, specify the input video file. The Intelligent Highlights feature currently does not support AWS S3 as an input source.
3. Then, add an Intelligent Analysis node at the "Process Input File" step.
﻿
In the Intelligent Analysis Settings drawer that appears, select the Intelligent Highlights preset template (template ID: 26). You can enable Extended Parameters under More Settings and pass extended parameters as described in this documentation to specify a highlight scenario and achieve better highlights.
﻿
4. Finally, after specifying the output video storage path, click Create to initiate the task.
Method 3: Automatically Triggering a Task
If you want to upload a video file to the COS bucket and achieve automatic Smart Erase according to preset parameters, you can:
1. Click Save the Orchestration when creating the task, and configure the Trigger Bucket, Trigger Directory, and other parameters in the pop-up window.
﻿
﻿
﻿
2. Then, go to the Offline Orchestration, find the newly created orchestration, and enable it by clicking the Enable button. As for the subsequent new video files in the Trigger Directory, tasks will be automatically triggered according to the preset process and parameters in the orchestration, and the processed video files will be saved to the output path configured by the orchestration.
Note:
It takes 3-5 minutes to take effect after the orchestration is successfully enabled.
﻿
Querying Task Results
Querying Results in the Console
1. You can check the task status on the Task Management page in the console. When the subtask status is "Successful":
﻿
2. Click Task Details JSON to view the output result and output file path in the output message.
﻿
3. If you use COS as the output path, go to the COS Bucket > Output Bucket page in the MPS console to find your output directory. Files prefixed with hht in this directory are the Intelligent Highlights output files, including the video and cover image for each highlight clip.
Note:
Text content, such as titles and summaries, is not output to the bucket. It must be retrieved via event callback or API queries.
﻿
Event Notification Callback
When you use ProcessMedia to initiate an MPS task, you can configure the event callback via the TaskNotifyConfig parameter. After the task is completed, the task result will be returned via the configured callback. You can parse the event notification result using ParseNotification.
Querying Task Results by an API
After you initiate an MPS task with ProcessMedia, a task ID (TaskId) is returned, for example: 24000022-WorkflowTask-b20a8exxxxxxx1tt110253 or 24000022-ScheduleTask-774f101xxxxxxx1tt110253. Call the DescribeTaskDetail API and pass the task ID to obtain the task result. You need to parse the WorkflowTask ->AiAnalysisResultSet field to obtain the task result.
Initiating a Live Stream Processing Task
Using Preset Template No. 26 for Intelligent Analysis
To use Intelligent Highlights, you need to initiate a task using preset template No. 26 under "Intelligent Analysis" in MPS. Custom templates are not supported for highlight parameter adjustment. To specify an Intelligent Highlights edition or video scenario, you need to pass extended parameters when initiating a task. For details, see Highlight Scenarios & Examples of Extended Parameters below.
Specifying Live Streaming Highlight Scenarios & Examples of Extended Parameters
Currently, only preset template No. 26 can be used to initiate Intelligent Highlights tasks. Custom templates are not supported for highlight parameter adjustment. To specify an Intelligent Highlights edition or video scenario, you need to pass extended parameters when initiating a task. The following are common live streaming highlight scenarios and corresponding examples of extended parameters.
Note:
This section describes the live stream processing scenarios supported by Intelligent Highlights, along with corresponding examples of extended parameters. For instructions on how to initiate a task and pass extended parameters, see the description about initiating a processing task below. For detailed field definitions of extended parameters, see the appendix below.
The example parameters may not produce optimal highlights. It is recommended to fine-tune them based on the video characteristics. For professional support, please contact us.
Broadcast and New Media Live Streaming
{"hht":{"force_cls":10010,"scenario":"news"}, "des":{"sent_threshold":200,"time_threshold":300},"live":{"ssrf_ignore":true, "caller":"live"}}
Online Education Live Streaming
{"hht":{"force_cls":10010,"scenario":"education"}, "des":{"sent_threshold":200,"time_threshold":300},"live":{"ssrf_ignore":true}}
Live Shopping
{"hht":{"force_cls":10010,"scenario":"E-commerce"}, "des":{"sent_threshold":200,"time_threshold":300},"live":{"ssrf_ignore":true}}
Financial Live Streaming
{"hht":{"force_cls":10010,"scenario":"finance"}, "des":{"sent_threshold":250,"time_threshold":600},"live":{"ssrf_ignore":true}}
Football Match Live Streaming
{"hht":{"force_cls":4001,"football_close_replay":1},"live":{"ssrf_ignore":true,"caller":"live"}}
Initiating a Task via an API
Note:
Currently, live stream processing tasks can only be initiated via an API.
Initiating a Request via API Explorer
For details on calling an API to initiate a live stream processing task, see ProcessLiveStream. In the document, click API Explorer Debugging in the description to go to the page, fill in the required parameters, and initiate an online call.
Set Definition to 26, which specifies preset template No. 26 for intelligent analysis to initiate a highlight task. ExtendedParameter is used to specify a highlight scenario. For parameter examples, see Specifying Live Streaming Highlight Scenarios & Examples of Extended Parameters above. Note that when using API Explorer, you can enter JSON directly in ExtendedParameter without escaping.
﻿
﻿
﻿
Initiating a Request via an API
Send a POST request directly to Tencent Cloud. Note that when calling the API directly, you need to escape the JSON string in ExtendedParameter. Below is a sample request:
{
    "Url": "http://mps-pull.test.org/live/test.flv",
    "TaskNotifyConfig": {
        "NotifyType": "URL",
        "NotifyUrl": "http://test.cloud.com/callback"
    },
    "OutputStorage": {
        "Type": "COS",
        "CosOutputStorage": {
            "Bucket": "my-mpsdata-12345xxx99",
            "Region": "ap-guangzhou"
        }
    },
    "OutputDir": "/output/",
    "AiAnalysisTask": {
        "Definition": 26,
        "ExtendedParameter": "{\\"hht\\":{\\"top_clip\\":10, \\"force_cls\\":5003, \\"need_vad\\":1, \\"threshold\\":0.9, \\"merge_time\\":60, \\"merge_type\\":0, \\"res_save_type\\":1}}"
    }
}
Receiving Task Callback Results
Refer to Parse Live Stream Processing Results to parse AiAnalysisResultInfo fields.
Task Management Protocol
Refer to Task Management Documentation to manage the initiated tasks.
Appendix: Extended Parameter Field Description
Parameter
Required
Type
Description
top_clip
No
int
Selects the highlight clips with the highest confidence. Default value is 5.
Example: \\"top_clip\\":10 – Outputs up to 10 highlight clips with the highest confidence.
force_cls
No
int
Specify the highlights category:
10010: Short dramas and TV series
4001: Football
4002: Basketball
1001: Honor of Kings
100101: Honor of Kings Competition
1003: League of Legends
10020: Large model-based highlights, suitable for custom video scenarios, VLOG videos, and landscape videos
need_vad
No
int
VAD is used to identify the end of a sentence in video. VAD extension allows complete video speech, enabled by default.
1: Use VAD
0: Do not use
threshold
No
float
Confidence threshold. The clips below the threshold are filtered out, and each type of highlights has default threshold settings.
Remarks: You are advised not to set this parameter for the first time.
res_save_type
No
int
Whether to save the results, which are saved by default.
1: Save results
0: Output the time period only
output_pattern
No
string
Output video naming format. {} indicates placeholder.
{year}-{month}-{day}-{hour}-{minute}-{second}_{start_dts}-{end_dts}-{timestamp}-{session}.mp4
Default output format:
hht-{year}{month}{day}{hour}{minute}-{session}-{timestamp}-index.mp4
image_pattern
No
string
image-{start_dts}.jpg
The parameters that can be used as placeholders are the same as above.
Default output format:
hht-{year}{month}{day}{hour}{minute}-{session}-{timestamp}-index.jpg
merge_type
No
int
Note: Only available in offline scenarios. Do not merge for the default value 5003, and merge in other scenarios.
Whether to merge the results into one video:
1: Merge (top_clip parameter does not take effect)
0: Do not merge (merge_time parameter does not take effect)
merge_time
No
int
Note: Only available in offline scenarios. The default value 5003 is the actual output and the time shall not be longer than one hour in other scenarios.
Specify the video output length when merging into one video.
prompts
No
Object
Prompt definition list. You can specify prompts to customize the output. For usage, refer to the large model examples.
scenario
No
string
Specifies a scenario. This parameter is valid only when force_cls is 10020.
model_segment_limit
No
Array
Note: This parameter controls the model output duration. The actual output is ultimately determined by the model.
Output video length limit. This value is passed to the model as a reference and is not mandatory. This parameter is valid only when force_cls is 10020. Example:
"model_segment_limit":[3,6]   // Suggests that the large model output segments be 3 to 6 seconds long.
prompts Structure
Parameter
Required
Type
Description
multimodal_prompt
No
string
Multimodal model prompt.

Was this page helpful?

You can also Contact Sales or Submit a Ticket for help.

Yes

Feedback

Processing Type	Intelligent Highlights Edition	Description	Supported Video Scenarios	Billing
Process offline files	Intelligent Highlights - Large Model Edition	This edition leverages large model-based video understanding to automatically analyze video content and extract key scenes and highlight moments. Video understanding prompts are customizable.	Custom scenarios Panoramic camera and VLOG videos Short dramas, films, and TV series	Charged under "Intelligent Highlights - Large Model Edition".
		Intelligent Highlights - Advanced Edition	Advanced algorithms are used in this edition, primarily tailored for sports events and esports videos.	Football matches Basketball matches Esports videos General scenarios	Charged under "Intelligent Highlights - Advanced Edition".
		Intelligent Highlights - Basic Edition	As of December 2025, this edition has been decommissioned and is now only available to existing customers.			Charged under "Intelligent Highlights - Basic Edition".
Process live streams	Intelligent Highlights - Live Stream Processing	Built on large model-based video understanding, this edition provides targeted model optimizations for live streaming scenarios.	Broadcast and new media live streaming Online education live streaming Live shopping Financial live streaming Football/basketball match live streaming	Charged under "Intelligent Highlights - Large Model Edition".

Parameter	Required	Type	Description
top_clip	No	int	Selects the highlight clips with the highest confidence. Default value is 5. Example: `\\"top_clip\\":10` – Outputs up to 10 highlight clips with the highest confidence.
force_cls	No	int	Specify the highlights category: 10010: Short dramas and TV series 4001: Football 4002: Basketball 1001: Honor of Kings 100101: Honor of Kings Competition 1003: League of Legends 10020: Large model-based highlights, suitable for custom video scenarios, VLOG videos, and landscape videos
need_vad	No	int	VAD is used to identify the end of a sentence in video. VAD extension allows complete video speech, enabled by default. 1: Use VAD 0: Do not use
threshold	No	float	Confidence threshold. The clips below the threshold are filtered out, and each type of highlights has default threshold settings. Remarks: You are advised not to set this parameter for the first time.
res_save_type	No	int	Whether to save the results, which are saved by default. 1: Save results 0: Output the time period only
output_pattern	No	string	Output video naming format. {} indicates placeholder. {year}-{month}-{day}-{hour}-{minute}-{second}_{start_dts}-{end_dts}-{timestamp}-{session}.mp4 Default output format: hht-{year}{month}{day}{hour}{minute}-{session}-{timestamp}-index.mp4
image_pattern	No	string	image-{start_dts}.jpg The parameters that can be used as placeholders are the same as above. Default output format: hht-{year}{month}{day}{hour}{minute}-{session}-{timestamp}-index.jpg
merge_type	No	int	Note: Only available in offline scenarios. Do not merge for the default value 5003, and merge in other scenarios. Whether to merge the results into one video: 1: Merge (top_clip parameter does not take effect) 0: Do not merge (merge_time parameter does not take effect)
merge_time	No	int	Note: Only available in offline scenarios. The default value 5003 is the actual output and the time shall not be longer than one hour in other scenarios. Specify the video output length when merging into one video.
prompts	No	Object	Prompt definition list. You can specify prompts to customize the output. For usage, refer to the large model examples.
scenario	No	string	Specifies a scenario. This parameter is valid only when force_cls is 10020.
model_segment_limit	No	Array	Note: This parameter controls the model output duration. The actual output is ultimately determined by the model. Output video length limit. This value is passed to the model as a reference and is not mandatory. This parameter is valid only when force_cls is 10020. Example: "model_segment_limit":[3,6] // Suggests that the large model output segments be 3 to 6 seconds long.

tencent cloud

Overview

Highlight Scenarios and Billing

Demo

Pre-integration Prerequisites

Initiating an Offline Video File Processing Task

Using Preset Template No. 26 for Intelligent Analysis

Specifying Offline Highlight Scenarios & Examples of Extended Parameters

Intelligent Highlights - Large Model Edition

Custom Scenarios

Panoramic Camera and VLOG Videos

Short Dramas, Films, and TV Series

Intelligent Highlights - Advanced Edition

Football Matches

Basketball Matches

Method 1: Initiating a Task via an API

1. Quick Verification with API Explorer

﻿2. Initiating via API

Method 2: Console Task Initiation (Zero-Code Automatic Generation)

Method 3: Automatically Triggering a Task

Querying Task Results

Querying Results in the Console

Event Notification Callback

Querying Task Results by an API

Initiating a Live Stream Processing Task

Using Preset Template No. 26 for Intelligent Analysis

Specifying Live Streaming Highlight Scenarios & Examples of Extended Parameters

Broadcast and New Media Live Streaming

Online Education Live Streaming

Live Shopping

Financial Live Streaming

Football Match Live Streaming

Initiating a Task via an API

Initiating a Request via API Explorer

Initiating a Request via an API

Receiving Task Callback Results

Task Management Protocol

Appendix: Extended Parameter Field Description

prompts Structure

2. Initiating via API