tencent cloud

Video Content Analysis
Download PDF
Last updated: 2023-03-22 14:46:48
Video Content Analysis
Last updated: 2023-03-22 14:46:48
Download PDF
Audio/Video content analysis is an offline task that intelligently analyzes audio/video content with the aid of AI. It intelligently gives suggestions for video categorization, labeling, and thumbnail generation to help video platforms manage videos more accurately and efficiently.
Audio/Video content analysis can do the following:
Feature
Description
Intelligent categorization
Gives suggestions on classifying videos into over 10 categories, including:
news, entertainment, gaming, technology, food, sports, travel, animation, dance, music, movies & TV, and automobiles.
Intelligent labeling
Gives suggestions on labeling videos. Currently, more than 3,000 labels are supported, including:
gaming, transportation, musician, racing, pet, drums, bicycle, WOW, computer, school, and jacket.
Intelligent thumbnail generation
Captures one or more screenshots of a video as the recommended cover.
Intelligent labeling by frame
Gives suggestions on labeling each frame of a video. Currently, over 1,000 labels are supported, including:
contemporary dance, water sports, steak, baby, kitten, annual plant, destroyer, comics, lawn, wedding dress, function room, and passport.

Audio/Video Analysis Template

You can use audio/video analysis parameters (templates) to specify the operations an audio/video analysis task performs:
Whether to enable intelligent categorization.
Whether to enable intelligent labeling.
Whether to enable intelligent thumbnail generation.
Whether to enable intelligent labeling by frame.
VOD provides preset audio/video analysis templates for common parameter combinations. You can also use a server API to create and manage custom templates.

Initiating a Task

You can initiate an audio/video analysis task by calling a server API, via the console, or by specifying the task when uploading videos. For details, see Task Initiation.
Below are the details:
Initiate a task by calling a server API: Call ProcessMedia, setting Definition in the request parameter AiAnalysisTask to the ID of the audio/video content analysis template.
Initiate a task via the console: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask), and use it to process videos in the console.
Specify a task when uploading videos from the server: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask). When calling ApplyUpload, set the parameter procedure to the task flow.
Specify a task when uploading videos from a client: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask). When generating a signature for upload, set the parameter procedure to the task flow.
Specify a task when uploading videos via the console: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask). When uploading videos via the console, select Auto-processing after upload and choose the task flow.

Getting the Result

After initiating an audio/video content analysis task, you can wait for the result notification asynchronously or perform a task query synchronously to get the task execution result. Below is an example of getting the result notification in normal callback mode after a content analysis task is initiated (the fields with null value are omitted):
{
    "EventType":"ProcedureStateChanged",
    "ProcedureStateChangeEvent":{
        "TaskId":"1256768367-Procedure-2e1af2456351812be963e309cc133403t0",
        "Status":"FINISH",
        "FileId":"5285890784246869930",
        "FileName":"Animal World",
        "FileUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/AtUCmy6gmIYA.mp4",
        "MetaData":{
            "AudioDuration":60,
            "AudioStreamSet":[
                {
                    "Bitrate":383854,
                    "Codec":"aac",
                    "SamplingRate":48000
                }
            ],
            "Bitrate":1021028,
            "Container":"mov,mp4,m4a,3gp,3g2,mj2",
            "Duration":60,
            "Height":480,
            "Rotate":0,
            "Size":7700180,
            "VideoDuration":60,
            "VideoStreamSet":[
                {
                    "Bitrate":637174,
                    "Codec":"h264",
                    "Fps":23,
                    "Height":480,
                    "Width":640
                }
            ],
            "Width":640
        },
        "AiAnalysisResultSet":[
            {
                "Type":"Classification",
                "ClassificationTask":{
                    "Status":"SUCCESS",
                    "ErrCode":0,
                    "Message":"",
                    "Input":{
                        "Definition":10
                    },
                    "Output":{
                        "ClassificationSet":[
                            {
                                "Classification":"Animals",
                                "Confidence":80
                            },
                            {
                                "Classification":"Travel",
                                "Confidence":34
                            }
                        ]
                    }
                }
            },
            {
                "Type":"Cover",
                "CoverTask":{
                    "Status":"SUCCESS",
                    "ErrCode":0,
                    "Message":"",
                    "Input":{
                        "Definition":10
                    },
                    "Output":{
                        "CoverSet":[
                            {
                                "CoverUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/xxx1.jpg",
                                "Confidence":79
                            },
                            {
                                "CoverUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/xxx2.jpg",
                                "Confidence":70
                            },
                            {
                                "CoverUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/xxx3.jpg",
                                "Confidence":66
                            }
                        ]
                    }
                }
            },
            {
                "Type":"Tag",
                "TagTask":{
                    "Status":"SUCCESS",
                    "ErrCode":0,
                    "Message":"",
                    "Input":{
                        "Definition":10
                    },
                    "Output":{
                        "TagSet":[
                            {
                                "Tag":"Horse",
                                "Confidence":34
                            },
                            {
                                "Tag":"Bird",
                                "Confidence":27
                            },
                            {
                                "Tag":"Plant",
                                "Confidence":13
                            },
                            {
                                "Tag":"Beach",
                                "Confidence":11
                            }
                        ]
                    }
                }
            }
        ],
        "TasksPriority":0,
        "TasksNotifyMode":""
    }
}
In the callback result, ProcedureStateChangeEvent.AiAnalysisResultSet contains three types of analysis results, which are video categorization (Classification), thumbnail generation (Cover), and labeling (Tag).
For video categorization (Classification), the category with the highest confidence score (Output.ClassificationSet) is Travel.
For thumbnail generation (Cover), three thumbnails (CoverSet) are recommended. CoverUrl indicates the download URL of each thumbnail.
For labeling (Tag), four labels (Output.TagSet) are recommended, which are listed by confidence score in descending order.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No

Feedback