tencent cloud

Feedback

Facial Keypoint Detection

Last updated: 2023-05-18 16:30:41
    Detects when a face is partially captured or concealed or when there are multiple faces; recognizes 256 facial keypoints.

    Index Image for 256 Facial Keypoints

    
    
    

    iOS API Description

    Integration guide for iOS

    For directions on how to integrate the Tencent Effect SDK for iOS, see the integration guide for iOS.

    Registering an Xmagic listener

    /// @brief The SDK event listener
    /// @param listener The listener for SDK events, including AI events, tips, and resource events.
    - (void)registerSDKEventListener:(id<YTSDKEventListener> _Nullable)listener;

    YTSDKEventListener callback description

    #pragma mark - Event callback APIs
    /// @brief SDK event callback APIs
    @protocol YTSDKEventListener <NSObject>
    /// @brief `YTDataUpdate` event callback
    /// @param event: Callback in NSString* format
    - (void)onYTDataEvent:(id _Nonnull)event;
    /// @brief AI event callback
    /// @param event: Callback in dict format
    - (void)onAIEvent:(id _Nonnull)event;
    /// @brief Tip event callback
    /// @param event: Callback in dict format
    - (void)onTipsEvent:(id _Nonnull)event;
    /// @brief Resource pack event callback
    /// @param event: Callback in string format
    - (void)onAssetEvent:(id _Nonnull)event;
    @end
    After the callbacks are configured successfully on Version 2.6.0 and earlier versions, the SDK will send a callback of facial data for each video frame.
    - (void)onYTDataEvent:(id _Nonnull)event;
    After the callbacks are configured successfully on Version 3.0.0, the SDK will send a callback of facial data for each video frame.
    - (void)onAIEvent:(id _Nonnull)event;
    //在onAIEvent方法中可通过下边方法可以获取到数据
    NSDictionary *eventDict = (NSDictionary *)event;
    if (eventDict[@"ai_info"] != nil) {
    NSLog(@"ai_info %@",eventDict[@"ai_info"]);
    }
    The data returned is in JSON format and includes the following fields (for details about the 256 facial keypoints, see the illustration above):
    /// @note The list of field descriptions
    /**
    | Field | Type | Value Range | Description |
    | :---- | :---- |:---- | :---- |
    | trace_id | int | [1,INF) | The face ID. If the faces obtained continuously from a video stream have the same face ID, they belong to the same person. |
    | face_256_point | float | [0,screenWidth] or [0,screenHeight] | 512 values in total for 256 facial keypoints. (0,0) is the top-left corner of the screen. |
    | face_256_visible | float | [0,1] | Visibility of the 256 facial keypoints. |
    | out_of_screen | bool | true/false | Whether the face is outside of the screen view. |
    | left_eye_high_vis_ratio | float | [0,1] | The percentage of keypoints with high visibility for the left eye. |
    | right_eye_high_vis_ratio | float | [0,1] | The percentage of keypoints with high visibility for the right eye. |
    | left_eyebrow_high_vis_ratio | float | [0,1] | The percentage of keypoints with high visibility for the left eyebrow. |
    | right_eyebrow_high_vis_ratio | float | [0,1] | The percentage of keypoints with high visibility for the right eyebrow. |
    | mouth_high_vis_ratio | float | [0,1] | The percentage of keypoints with high visibility for the mouth. |
    **/
    - (void)onYTDataEvent:(id _Nonnull)event;

    Android API Description

    Integration guide for Android

    For directions on how to integrate the Tencent Effect SDK for Android, see the integration guide for Android.

    Registering an Xmagic listener

    This API is used to configure the callback of facial keypoints and other data.
    Version 2.6.0 and earlier versions use the following method.
    void setYTDataListener(XmagicApi.XmagicYTDataListener ytDataListener)
    
    
    public interface XmagicYTDataListener {
    void onYTDataUpdate(String data)
    }
    Version 3.0.0 uses the following method.
    void setAIDataListener(XmagicApi.OnAIDataListener aiDataListener)
    
    public interface OnAIDataListener {
    void onFaceDataUpdated(List<TEFaceData> faceDataList);
    void onHandDataUpdated(List<TEHandData> handDataList);
    void onBodyDataUpdated(List<TEBodyData> bodyDataList);
    void onAIDataUpdated(String data); //This method is a new method added in version 3.0.0, and the data structure is consistent with the XmagicYTDataListener interface used in previous versions.
    }
    onYTDataUpdate and onAIDataUpdated returns a JSON string structure that contains the information of up to 5 faces:
    {
    "face_info":[{
    "trace_id":5,
    "face_256_point":[
    180.0,
    112.2,
    ...
    ],
    "face_256_visible":[
    0.85,
    ...
    ],
    "out_of_screen":true,
    "left_eye_high_vis_ratio:1.0,
    "right_eye_high_vis_ratio":1.0,
    "left_eyebrow_high_vis_ratio":1.0,
    "right_eyebrow_high_vis_ratio":1.0,
    "mouth_high_vis_ratio":1.0
    },
    ...
    ]
    }

    Fields

    Field
    Type
    Value Range
    Description
    trace_id
    int
    [1,INF)
    The face ID. If the faces obtained continuously from a video stream have the same face ID, they belong to the same person.
    face_256_point
    float
    [0,screenWidth] or [0,screenHeight]
    512 values in total for 256 facial keypoints. (0,0) is the top-left corner of the screen.
    face_256_visible
    float
    [0,1]
    Visibility of the 256 facial keypoints.
    out_of_screen
    bool
    true/false
    Whether the face is outside of the screen view.
    left_eye_high_vis_ratio
    float
    [0,1]
    The percentage of keypoints with high visibility for the left eye.
    right_eye_high_vis_ratio
    float
    [0,1]
    The percentage of keypoints with high visibility for the right eye.
    left_eyebrow_high_vis_ratio
    float
    [0,1]
    The percentage of keypoints with high visibility for the left eyebrow.
    right_eyebrow_high_vis_ratio
    float
    [0,1]
    The percentage of keypoints with high visibility for the right eyebrow.
    mouth_high_vis_ratio
    float
    [0,1]
    The percentage of keypoints with high visibility for the mouth.

    Parameters

    Parameters
    Description
    XmagicApi.XmagicYTDataListener ytDataListener
    Callback function implementation class.
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support