tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Video on Demand
    Documentation
    Download PDF

    Video Content Analysis

    Last updated: 2023-03-22 14:46:48
    Download PDF
      Audio/Video content analysis is an offline task that intelligently analyzes audio/video content with the aid of AI. It intelligently gives suggestions for video categorization, labeling, and thumbnail generation to help video platforms manage videos more accurately and efficiently.
      Audio/Video content analysis can do the following:
      Feature
      Description
      Intelligent categorization
      Gives suggestions on classifying videos into over 10 categories, including:
      news, entertainment, gaming, technology, food, sports, travel, animation, dance, music, movies & TV, and automobiles.
      Intelligent labeling
      Gives suggestions on labeling videos. Currently, more than 3,000 labels are supported, including:
      gaming, transportation, musician, racing, pet, drums, bicycle, WOW, computer, school, and jacket.
      Intelligent thumbnail generation
      Captures one or more screenshots of a video as the recommended cover.
      Intelligent labeling by frame
      Gives suggestions on labeling each frame of a video. Currently, over 1,000 labels are supported, including:
      contemporary dance, water sports, steak, baby, kitten, annual plant, destroyer, comics, lawn, wedding dress, function room, and passport.

      Audio/Video Analysis Template

      You can use audio/video analysis parameters (templates) to specify the operations an audio/video analysis task performs:
      Whether to enable intelligent categorization.
      Whether to enable intelligent labeling.
      Whether to enable intelligent thumbnail generation.
      Whether to enable intelligent labeling by frame.
      VOD provides preset audio/video analysis templates for common parameter combinations. You can also use a server API to create and manage custom templates.

      Initiating a Task

      You can initiate an audio/video analysis task by calling a server API, via the console, or by specifying the task when uploading videos. For details, see Task Initiation.
      Below are the details:
      Initiate a task by calling a server API: Call ProcessMedia, setting Definition in the request parameter AiAnalysisTask to the ID of the audio/video content analysis template.
      Initiate a task via the console: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask), and use it to process videos in the console.
      Specify a task when uploading videos from the server: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask). When calling ApplyUpload, set the parameter procedure to the task flow.
      Specify a task when uploading videos from a client: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask). When generating a signature for upload, set the parameter procedure to the task flow.
      Specify a task when uploading videos via the console: Call the server API CreateProcedureTemplate to create an audio/video analysis task flow (MediaProcessTask.AiAnalysisTask). When uploading videos via the console, select Auto-processing after upload and choose the task flow.

      Getting the Result

      After initiating an audio/video content analysis task, you can wait for the result notification asynchronously or perform a task query synchronously to get the task execution result. Below is an example of getting the result notification in normal callback mode after a content analysis task is initiated (the fields with null value are omitted):
      {
          "EventType":"ProcedureStateChanged",
          "ProcedureStateChangeEvent":{
              "TaskId":"1256768367-Procedure-2e1af2456351812be963e309cc133403t0",
              "Status":"FINISH",
              "FileId":"5285890784246869930",
              "FileName":"Animal World",
              "FileUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/AtUCmy6gmIYA.mp4",
              "MetaData":{
                  "AudioDuration":60,
                  "AudioStreamSet":[
                      {
                          "Bitrate":383854,
                          "Codec":"aac",
                          "SamplingRate":48000
                      }
                  ],
                  "Bitrate":1021028,
                  "Container":"mov,mp4,m4a,3gp,3g2,mj2",
                  "Duration":60,
                  "Height":480,
                  "Rotate":0,
                  "Size":7700180,
                  "VideoDuration":60,
                  "VideoStreamSet":[
                      {
                          "Bitrate":637174,
                          "Codec":"h264",
                          "Fps":23,
                          "Height":480,
                          "Width":640
                      }
                  ],
                  "Width":640
              },
              "AiAnalysisResultSet":[
                  {
                      "Type":"Classification",
                      "ClassificationTask":{
                          "Status":"SUCCESS",
                          "ErrCode":0,
                          "Message":"",
                          "Input":{
                              "Definition":10
                          },
                          "Output":{
                              "ClassificationSet":[
                                  {
                                      "Classification":"Animals",
                                      "Confidence":80
                                  },
                                  {
                                      "Classification":"Travel",
                                      "Confidence":34
                                  }
                              ]
                          }
                      }
                  },
                  {
                      "Type":"Cover",
                      "CoverTask":{
                          "Status":"SUCCESS",
                          "ErrCode":0,
                          "Message":"",
                          "Input":{
                              "Definition":10
                          },
                          "Output":{
                              "CoverSet":[
                                  {
                                      "CoverUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/xxx1.jpg",
                                      "Confidence":79
                                  },
                                  {
                                      "CoverUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/xxx2.jpg",
                                      "Confidence":70
                                  },
                                  {
                                      "CoverUrl":"http://1256768367.vod2.myqcloud.com/xxx/xxx/xxx3.jpg",
                                      "Confidence":66
                                  }
                              ]
                          }
                      }
                  },
                  {
                      "Type":"Tag",
                      "TagTask":{
                          "Status":"SUCCESS",
                          "ErrCode":0,
                          "Message":"",
                          "Input":{
                              "Definition":10
                          },
                          "Output":{
                              "TagSet":[
                                  {
                                      "Tag":"Horse",
                                      "Confidence":34
                                  },
                                  {
                                      "Tag":"Bird",
                                      "Confidence":27
                                  },
                                  {
                                      "Tag":"Plant",
                                      "Confidence":13
                                  },
                                  {
                                      "Tag":"Beach",
                                      "Confidence":11
                                  }
                              ]
                          }
                      }
                  }
              ],
              "TasksPriority":0,
              "TasksNotifyMode":""
          }
      }
      In the callback result, ProcedureStateChangeEvent.AiAnalysisResultSet contains three types of analysis results, which are video categorization (Classification), thumbnail generation (Cover), and labeling (Tag).
      For video categorization (Classification), the category with the highest confidence score (Output.ClassificationSet) is Travel.
      For thumbnail generation (Cover), three thumbnails (CoverSet) are recommended. CoverUrl indicates the download URL of each thumbnail.
      For labeling (Tag), four labels (Output.TagSet) are recommended, which are listed by confidence score in descending order.
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy