This document describes how to integrate with Game Multimedia Engine (GME) on macOS.
Note:
This document applies to GME SDK version 2.9.
Considerations
GME provides two services: voice chat service and voice message and speech-to-text service, both of which rely on key APIs such as Init and Poll.
Note on Init API
If you need to use voice chat and voice message services at the same time, you only need to call Init API once.\nBilling will not start after initialization. After you call Entering a voice chat room to enter the room successfully, the billing will start. Directions
Important notes
Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, QAVError.OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
APIs
@class ITMGRoom;//Room APIs
@class ITMGAudioCtrl;//Audio APIs
@class ITMGAudioEffectCtrl;//Sound effect, accompaniment APIs
Key APIs
Before the initialization, the SDK is in the uninitialized status, and you need to initialize it through the Init API before you can use the voice chat and speech-to-text services.
You need to call the Init API before calling any APIs of GME.
If you have any questions when using the service, please see General FAQs. Getting singleton
To use the voice feature, get the ITMGContext object first.
Function prototype
ITMGContext ITMGDelegate <NSObject>
Sample code
ITMGContext* _context = [ITMGContext GetInstance];
_context.TMGDelegate =self;
Message delivery
The API callback messages is processed in OnEvent. For the message type, please see ITMG_MAIN_EVENT_TYPE. The message content is a dictionary for parsing the API callback contents.
Function prototype
- (void)OnEvent:(ITMG_MAIN_EVENT_TYPE)eventType data:(NSDictionary*)data;
Sample code
-(void)OnEvent:(ITMG_MAIN_EVENT_TYPE)eventType data:(NSDictionary *)data{
NSLog(@"OnEvent:%lu,data:%@",(unsigned long)eventType,data);
switch (eventType) {
//Identify `eventType`
}
}
This API is used to initialize the GME service. It is recommended to call it when initializing the application. No fee is incurred for calling this API.
The openID uniquely identifies a user with the rules stipulated by the application developer and unique in the application (currently, only INT64 is supported).
Note:
The SDK must be initialized before a user can enter a voice chat room.
The Init API must be called in the same thread with other APIs. It is recommended to call all APIs in the main thread.
Function prototype
-(int)InitEngine:(NSString*)sdkAppID openID:(NSString*)openID;
|
| | |
| | OpenId can only be in Int64 type, which is passed after being converted to a string.
|
Returned values
|
| Initialized SDK successfully. |
QAV_ERR_SDK_NOT_FULL_UPDATE= 7015 | Checks whether the SDK file is complete. It is recommended to delete it and then import the SDK again. |
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
If this error is reported during integration, please check the integrity and version of the SDK file as prompted.
If this error is returned after executable file export, please ignore it and try to avoid displaying it in the UI.
Sample code
_openId = _userIdText.text;
_appId = _appIdText.text;
[[ITMGContext GetInstance] InitEngine:SDKAPPID openID:_openId];
Triggering event callback
Event callbacks can be triggered by periodically calling the Poll API in update. The Poll API should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run exceptionally.\nRefer to the EnginePollHelper.m file in Demo. Calling the `Poll` API periodically
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
Function prototype
Sample code
[[ITMGContext GetInstance] Poll];
Pausing the system
When a Pause event occurs in the system, the engine should also be notified for pause.\nIf you need to pause the audio when switching to the background, you can call the Pause API in the listening code used to switch to the background, and call the Resume API in the listening event used to resume the foreground.
Function prototype
Resuming the system
When a Resume event occurs in the system, the engine should also be notified for resumption. The Resume API only supports resuming voice chat.
Function prototype
This API is used to uninitialize the SDK to make it uninitialized. Switching accounts requires uninitialization.
Function prototype
Sample code
[[ITMGContext GetInstance] Uninit];
Voice chat flowchart
Voice Chat Room APIs
You should initialize and call the SDK to enter a room before voice chat can start.\nIf you have any questions when using the service, please see FAQs About Voice Chat. |
| Initializes authentication |
| |
| Indicates whether room entry is successful |
| |
| Modifies user's room audio type |
| Gets user's room audio type |
Voice chat room call flowchart
Entered the room successfully
If the room entry callback result is 0, the room entry is successful. The returned value of 0 from the EnterRoom API does not necessarily mean that the room entry is successful.
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, please use the backend deployment key as detailed in Authentication Key. Function prototype
@interface QAVAuthBuffer : NSObject
+ (NSData*) GenAuthBuffer:(unsigned int)appId roomId:(NSString*)roomId openID:(NSString*)openID key:(NSString*)key;
+ @end
|
| | AppId from the Tencent Cloud console.
|
| | Room ID, which can contain up to 127 characters. |
| | User ID, which is the same as openID during initialization. |
| | Permission key from the Tencent Cloud console. |
Sample code
#import "GMESDK/QAVAuthBuffer.h"
NSData* authBuffer = [QAVAuthBuffer GenAuthBuffer:SDKAPPID3RD.intValue roomId:_roomId openID:_openId key:AUTHKEY];
When a user enters a room with the generated authentication information, the ITMG_MAIN_EVENT_TYPE_ENTER_ROOM message will be received as a callback. Mic and speaker are not enabled by default after room entry. The returned value of AV_OK indicates a success.
The audio type of the room is determined by the first user to enter the room. After that, if a member in the room changes the room type, it will take effect for all members there. For example, if the first user entering the room uses the smooth sound quality, and the second user entering the room uses the HD sound quality, the room audio type of the second user will become smooth sound quality. Only when a member in the room calls the ChangeRoomType API, the audio type of the room will be changed.
Function prototype
-(int)EnterRoom:(NSString*) roomId roomType:(int)roomType authBuffer:(NSData*)authBuffer;
|
| | Room ID, which can contain up to 127 characters |
| | |
| | |
Sample code
[[ITMGContext GetInstance] EnterRoom:_roomId roomType:_roomType authBuffer:authBuffer];
Callback for room entry
After the user enters the room, the message ITMG_MAIN_EVENT_TYPE_ENTER_ROOM will be sent and identified in the OnEvent function for callback and processing. A successful callback means that the room entry is successful, and the billing starts.
Sample code
Sample code for processing the callback, including room entry and network disconnection events.
-(void)OnEvent:(ITMG_MAIN_EVENT_TYPE)eventType data:(NSDictionary *)data{
NSLog(@"OnEvent:%lu,data:%@",(unsigned long)eventType,data);
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
int result = ((NSNumber*)[data objectForKey:@"result"]).intValue;
NSString* error_info = [data objectForKey:@"error_info"];
//Receive the event of successful room entry
}
break;
}
}
Data details
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | | {"error_info":"waiting timeout, please check your network","result":0} |
If the network is disconnected, there will be a disconnected callback prompt ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT. At this time, the SDK will automatically reconnect, and the callback is ITMG_MAIN_EVENT_TYPE_RECONNECT_START. When the reconnection is successful, there will be a callback ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS.
Error codes
|
| Authentication failed: The AppID does not exist or is incorrect. An error occurred while authenticating the authbuff. Authentication expired. The OpenId does not meet the specification. |
| |
| The user was already in the process of entering a room but repeated this operation. It is recommended not to call the room entering API until the room entry callback is returned. |
| The user was already in the room and called the room entering API again. |
| Make sure that the SDK is initialized, OpenId complies with the rules, the APIs are called in the same thread, and the Poll API is called normally. |
This API is called to exit the current room. It is an async API. The returned value of AV_OK indicates a successful async delivery.
Note:
If there is a scenario in the application where room entry is performed immediately after room exit, you don't need to wait for the RoomExitComplete callback notification from the ExitRoom API during API call; instead, you can directly call the API.
Function prototype
Sample code
[[ITMGContext GetInstance] ExitRoom];
Callback for room exit
After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM.
Sample code
-(void)OnEvent:(ITMG_MAIN_EVENT_TYPE)eventType data:(NSDictionary *)data{
NSLog(@"OnEvent:%lu,data:%@",(unsigned long)eventType,data);
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_EXIT_ROOM:
{
// Receive the event of successful room exit
}
break;
}
}
Data details
|
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | | {"error_info":"","result":0} |
Determining whether user has entered room
This API is used to determine whether the user has entered a room. A bool-type value will be returned. The call is invalid during the process of room entry.
Function prototype
Sample code
[[ITMGContext GetInstance] IsRoomEntered];
Switching room
User can call this API to quickly switch the voice chat room after entering the room. After the room is switched, the device is not reset, that is, if the microphone is already enabled in this room, the microphone will keep enabled after the room is switched.
The callback for quickly switching rooms is ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM, and the fields are error_info and result.
API prototype
-(int) SwitchRoom:(NSString *)roomID authBuffer:(NSData*)authBuffer;
Type descriptions
|
| | |
| | Generates a new authentication with the ID of the room to enter |
Callback sample code
- (IBAction)swichRoom:(id)sender {
NSData* authBuffer = [QAVAuthBuffer GenAuthBuffer:_appId.intValue roomID:_roomIdText.text openID:_openId key:_key];
[[[ITMGContext GetInstance]GetRoom]SwitchRoom:_roomIdText.text authBuffer:authBuffer];
}
-(void)OnEvent:(ITMG_MAIN_EVENT_TYPE)eventType data:(NSDictionary *)data{
NSString* log =[NSString stringWithFormat:@"OnEvent:%d,data:%@", (int)eventType, data];
[self showLog:log];
NSLog(@"====%@====",log);
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM:
{
int result = ((NSNumber*)[data objectForKey:@"result"]).intValue;
NSString* log = nil;
if (result == QAV_OK) {
log = [NSString stringWithFormat:@"switch room success."];
} else {
log = [NSString stringWithFormat:@"switch room failed."];
}
[self showLog:log];
break;
}
}
}
Cross-room mic connection
Call this API to connect the microphones across rooms after entering the room. And the local user can communicate with the target OpenID user in the target room.
Example
User a is in room A, user b is in room B, and user a can talk with b through the cross-room API. When user c in room A speaks, users b and d in room B cannot hear. User c in room A can hear only the voice in room A and the voice of user b in room B but not other users in room B.
API prototype
-(int) StartRoomSharing:(NSString *)targetRoomID targetOpenID:(NSString *)targetOpenID authBuffer:(NSData*)authBuffer;
-(int) StopRoomSharing;
Type descriptions
|
| | ID of the room to connect mic |
| | The target OpenID to connect mic |
| | Reserved flag. You just need to enter NULL. |
Sample code
- (IBAction)shareRoom:(id)sender {
if(_shareRoomSwitch.isOn){
[[[ITMGContext GetInstance]GetRoom]StartRoomSharing:_shareRoomID.text targetOpenID:_shareOpenID.text authBuffer:NULL];
}else{
[[[ITMGContext GetInstance]GetRoom]StopRoomSharing];
}
}
}
Notifications of member room entry and speaking status
This API is used to obtain the user speaking in the room and display it in the UI, and to send a notification when someone entering or exiting the room.
Notification for this event will be sent only when the status changes. To get member status in real time, cache the notification when it is received at a higher layer. The event message is ITMG_MAIN_EVNET_TYPE_USER_UPDATE, where the data contains event_id and user_list. The event message will be identified in the OnEvent function.\nNotifications for audio events are subject to a threshold, and a notification will be sent only when this threshold is exceeded. The notification "A member has stopped sending audio packets” will be sent if no audio packets are received in more than two seconds. This event only returns the member speaking event, but not the specific volume level. If you need the specific volume levels of members in the room, use the GetVolumeById API.
|
| | |
| | |
ITMG_EVENT_ID_USER_HAS_AUDIO | A member sends audio packets. This event can be used to determine whether a user is speaking and display the voiceprint effect. It can be called together with getRecvStreamLevel. | |
ITMG_EVENT_ID_USER_NO_AUDIO | A member stops sending audio packets | |
Room member maintenance flowchart
Sample code
-(void)OnEvent:(ITMG_MAIN_EVENT_TYPE)eventType data:(NSDictionary *)data{
ITMG_EVENT_ID_USER_UPDATE event_id=((NSNumber*)[data objectForKey:@"event_id"]).intValue;
NSMutableArray* uses = [NSMutableArray arrayWithArray: [data objectForKey:@"user_list"]];
NSLog(@"OnEvent:%lu,data:%@",(unsigned long)eventType,data);
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_USER_UPDATE:
{
// Process
//Parse the parameter to get `event_id` and `user_list`
switch (eventID)
{
case ITMG_EVENT_ID_USER_ENTER:
// A member enters the room
break;
case ITMG_EVENT_ID_USER_EXIT:
// A member exits the room
break;
case ITMG_EVENT_ID_USER_HAS_AUDIO:
// A member sends audio packets
break;
case ITMG_EVENT_ID_USER_NO_AUDIO:
// A member stops sending audio packets
break;
}
break;
}
}
}
Data details
|
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | | {"event_id":0,"user_list":""} |
Muting in a room
This API is used to add an ID to the audio data blacklist. This operation blocks audio from someone and only applies to the local device. A returned value of 0 indicates the call is successful. Assume that users A, B, and C are all speaking using their mic in a room:
If A blocks C, A can only hear B;
If B blocks neither A nor C, B can hear both of them;
If C blocks neither A nor B, C can hear both of them.
This API is suitable for scenarios where a user is muted in a room.
Function prototype
ITMGContext GetAudioCtrl -(QAVResult)AddAudioBlackList:(NSString*)openID;
Sample code
[[[ITMGContext GetInstance]GetAudioCtrl ] AddAudioBlackList[id]];
Unmuting
This API is used to remove an ID from the audio data blocklist. A returned value of 0 indicates the call is successful.
Function prototype
-(QAVResult)RemoveAudioBlackList:(NSString*)openID;
|
| | ID to be unblocked openid |
Sample code
[[[ITMGContext GetInstance]GetAudioCtrl ] RemoveAudioBlackList[openId]];
Voice Chat Audio APIs
Notes on voice chat audio connection
The voice chat APIs can only be called after SDK initialization and room entry.\nWhen Enable/Disable Mic/Speaker is clicked on the UI, the following practices are recommended:
For most game applications, it is recommended to call the EnableMic and EnableSpeaker APIs, which is equivalent to calling the EnableAudioCaptureDevice/EnableAudioSend and EnableAudioPlayDevice/EnableAudioRecv APIs.
For other mobile applications (such as social networking applications), enabling/disabling a capturing device will restart both capturing and playback devices. If the application is playing back background music, it will also be interrupted. Playback will not be interrupted if the mic is enabled/disabled through control of upstreaming/downstreaming. Calling method: call EnableAudioCaptureDevice(true) and EnableAudioPlayDevice(true) once after room entry, and call EnableAudioSend/Recv to send/receive audio streams when Enable/Disable Mic is clicked.
For more information on how to release only a capturing or playback device, please see the EnableAudioCaptureDevice and EnableAudioPlayDevice.
Call the pause API to pause the audio engine and call the resume API to resume the audio engine.
Voice chat audio APIs
|
| |
| |
| Enables/disables capturing device |
IsAudioCaptureDeviceEnabled | Gets capturing device status |
| Enables/disables audio upstreaming |
| Gets audio upstreaming status |
| Gets real-time mic volume |
| Gets real-time audio upstreaming volume |
| |
| |
| |
| |
| Enables/disables playback device |
| Gets playback device status |
| Enables/disables audio downstreaming |
| Gets audio downstreaming status |
| Gets real-time speaker volume |
| Gets real-time downstreaming audio levels of other members in room |
| |
| |
| Enables/disables in-ear monitoring |
Voice Chat Capturing APIs
Enabling or disabling the microphone
This API is used to enable/disable the mic. Mic and speaker are not enabled by default after room entry.\nIf accompaniment is used, please call this API as instructed in Accompaniment in Voice Chat. EnableMic = EnableAudioCaptureDevice + EnableAudioSend
Function prototype
-(QAVResult)EnableMic:(BOOL)enable;
|
| | To enable the mic, set this parameter to YES; otherwise, set it to NO. |
Sample code
// Enable mic
[[[ITMGContext GetInstance] GetAudioCtrl] EnableMic:YES];
Getting the mic status
This API is used to get the mic status. The returned value 0 indicates that the mic is off, while 1 is on.
Function prototype
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetMicState];
Enabling or disabling capturing device
This API is used to enable/disable a capturing device. The device is not enabled by default after room entry.
This API can only be called after room entry. The device will be disabled automatically after room exit.
Operations such as permission application and volume type adjustment will generally be performed when a capturing device is enabled on a mobile device.
Function prototype
-(QAVResult)EnableAudioCaptureDevice:(BOOL)enabled;
|
| | To enable the capturing device, set this parameter to YES, otherwise set it to NO. |
Sample code
// Enable capturing device
[[[ITMGContext GetInstance]GetAudioCtrl ]EnableAudioCaptureDevice:enabled];
Getting the capturing device status
This API is used to get the status of a capturing device.
Function prototype
-(BOOL)IsAudioCaptureDeviceEnabled;
Sample code
BOOL IsAudioCaptureDevice = [[[ITMGContext GetInstance] GetAudioCtrl] IsAudioCaptureDeviceEnabled];
Enabling or disabling audio upstreaming
This API is used to enable/disable audio upstreaming. If a capturing device is already enabled, it will send captured audio data; otherwise, it will remain mute. For more information on how to enable/disable the capturing device, please see the EnableAudioCaptureDevice API.
Function prototype
-(QAVResult)EnableAudioSend:(BOOL)enable;
|
| | To enable audio upstreaming, set this parameter to YES; otherwise, set it to NO. |
Sample code
[[[ITMGContext GetInstance]GetAudioCtrl ]EnableAudioSend:enabled];
Getting audio upstreaming status
This API is used to get the status of audio upstreaming.
Function prototype
-(BOOL)IsAudioSendEnabled;
Sample code
BOOL IsAudioSend = [[[ITMGContext GetInstance] GetAudioCtrl] IsAudioSendEnabled];
Getting the real-time mic volume
This API is used to get the real-time mic volume. An int-type value in the range of 0-100 will be returned. It is recommended to call this API once every 20 ms.
This API is not applicable to the voice message service.
Function prototype
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetMicLevel];
Getting the real-time audio upstreaming volume
This API is used to get the local real-time audio upstreaming volume. An int-type value in the range of 0-100 will be returned.
This API is not applicable to the voice message service.
Function prototype
-(int)GetSendStreamLevel();
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetSendStreamLevel];
Setting the mic software volume
This API is used to set the mic volume. The corresponding parameter is volume, which is equivalent to attenuating or gaining the captured sound. 0 indicates that the audio is mute, while 100 indicates that the volume remains unchanged. The default value is 100.
This API is not applicable to the voice message service.
Function prototype
-(QAVResult)SetMicVolume:(int) volume;
|
| | Sets volume. Value range: 0-200 |
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] SetMicVolume:100];
Getting the mic software volume
This API is used to obtain the microphone volume. An "int" value is returned. Value 101 represents API SetMicVolume has not been called.
This API is not applicable to the voice message service.
Function prototype
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetMicVolume];
Voice Chat Playback APIs
Enabling or disabling the speaker
This API is used to enable/disable the speaker.\nIf accompaniment is used, please call this API as instructed in Accompaniment in Voice Chat. EnableSpeaker = EnableAudioPlayDevice + EnableAudioRecv
Function prototype
-(void)EnableSpeaker:(BOOL)enable;
|
| | To disable the speaker, set this parameter to NO, otherwise set it to YES. |
Sample code
// Enable the speaker
[[[ITMGContext GetInstance] GetAudioCtrl] EnableSpeaker:YES];
Getting the speaker status
This API is used to get the speaker status. 0 indicates that the speaker is off, while 1, on.
Function prototype
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetSpeakerState];
Enabling or disabling playback device
This API is used to enable/disable a playback device.
Function prototype
-(QAVResult)EnableAudioPlayDevice:(BOOL)enabled;
|
| | To disable a playback device, set this parameter to NO, otherwise set it to YES. |
Sample code
// Enable the playback device
[[[ITMGContext GetInstance]GetAudioCtrl ]EnableAudioPlayDevice:enabled];
Getting the playback device status
This API is used to get the status of a playback device.
Function prototype
-(BOOL)IsAudioPlayDeviceEnabled;
Sample code
BOOL IsAudioPlayDevice = [[[ITMGContext GetInstance] GetAudioCtrl] IsAudioPlayDeviceEnabled];
Enabling or disabling audio downstreaming
This API is used to enable/disable audio downstreaming. If a playback device is already enabled, it will play back audio data from other members in the room; otherwise, it will remain mute. For more information on how to enable/disable the playback device, please see the EnableAudioPlayDevice API.
Function prototype
-(QAVResult)EnableAudioRecv:(BOOL)enabled;
|
| | To enable audio downstreaming, set this parameter to YES; otherwise, set it to NO. |
Sample code
[[[ITMGContext GetInstance]GetAudioCtrl ]EnableAudioRecv:enabled];
Getting audio downstreaming status
This API is used to get the status of audio downstreaming.
Function prototype
-(BOOL)IsAudioRecvEnabled;
Sample code
BOOL IsAudioRecv = [[[ITMGContext GetInstance] GetAudioCtrl] IsAudioRecvEnabled];
Getting the real-time speaker volume
This API is used to get the real-time speaker volume level. An int-type value will be returned to indicate the volume level. It is recommended to call this API once every 20 ms.
Function prototype
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetSpeakerLevel];
Getting the real-time downstreaming audio levels of other members in room
This API is used to get the real-time audio downstreaming volume of other members in the room. An int-type value will be returned. Value range: 0-200.
Function prototype
-(int)GetRecvStreamLevel:(NSString*) openID;
|
| | openId of another member in the room
|
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetRecvStreamLevel:(NSString*) openId
Dynamic setting of the volume of a member in the room
This API is used to set the volume of a member in the room and only applies to the local device.
Function prototype
-(int) SetSpeakerVolumeByOpenID:(NSString *)openId volume:(int)volume;
Parameter description
|
| | OpenID that needs to adjust the volume |
| | Range: [0-200]; Default: 100 |
Getting the set volume percentage
Call this API to get volume set by SetSpeakerVolumeByOpenID
API prototype
-(int) GetSpeakerVolumeByOpenID:(NSString *)openId;
Returned values
API returns volume percentage set by OpenID, 100 by default.
Setting the speaker volume
This API is used to set the speaker volume.\nThe corresponding parameter is volume. 0 indicates that the audio is mute, while 100 indicates that the volume remains unchanged. The default value is 100.
Function prototype
-(QAVResult)SetSpeakerVolume:(int)vol;
|
| | Sets volume. Value range: 0-200 |
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] SetSpeakerVolume:100];
Getting the speaker volume
This API is used to get the speaker volume. An int-type value will be returned to indicate the volume. 101 indicates that the SetSpeakerVolume API has not been called.\n"Level" indicates the real-time volume, and "Volume" the speaker volume. The final volume = Level * Volume%. For example, if the "Level" is 100 and "Volume" is 60, the final volume is "60".
Function prototype
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] GetSpeakerVolume];
Advanced APIs
Enabling in-ear monitoring
This API is used to enable in-ear monitoring. You need to call EnableLoopBack+EnableSpeaker before you can hear your own voice.
Function prototype
-(QAVResult)EnableLoopBack:(BOOL)enable;
|
| | Specifies whether to enable in-ear monitoring. |
Sample code
[[[ITMGContext GetInstance] GetAudioCtrl] EnableLoopBack:YES];
Modifying user's room audio type
This API is used to modify a user's room audio type. For the result, please see the callback event. The event type is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE. The audio type of the room is determined by the first user to enter the room. After that, if a member in the room changes the room type, it will take effect for all members there.
Function prototype
-(int)ChangeRoomType:(int)nRoomType;
|
| | Target room type to be switched to. For room audio types, please see the EnterRoom API. |
Sample code
[[[ITMGContext GetInstance]GetRoom ]ChangeRoomType:_roomType];
Getting user's room audio type
This API is used to get a user's room audio type. The returned value is the room audio type. Value 0 indicates that an error occurred while getting the user's room audio type. For room audio types, please see the EnterRoom API.
Function prototype
Sample code
[[[ITMGContext GetInstance]GetRoom ]GetRoomType];
Callback for modifying the room type
After the room type is set, the event message ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE will be returned in the callback. The returned parameters include result, error_info, and new_room_type. The new_room_type represents the following information. The event message will be identified in the OnEvent function.
|
ITMG_ROOM_CHANGE_EVENT_ENTERROOM | | Indicates that the existing audio type is inconsistent with and changed to that of the entered room. |
ITMG_ROOM_CHANGE_EVENT_START | | Indicates that a user is already in the room and the audio type starts changing (e.g., calling the ChangeRoomType API to change the audio type). |
ITMG_ROOM_CHANGE_EVENT_COMPLETE | | Indicates that a user is already in the room and the audio type has been changed. |
ITMG_ROOM_CHANGE_EVENT_REQUEST | | Indicates that a room member calls the ChangeRoomType API to request a change of room audio type. |
Sample code
-(void)OnEvent:(ITMG_MAIN_EVENT_TYPE)eventType data:(NSDictionary *)data{
NSLog(@"OnEvent:%lu,data:%@",(unsigned long)eventType,data);
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE:
NSLog(@"ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE:%@ ",data);
int result = ((NSNumber*)[data objectForKey:@"result"]).intValue;
int newRoomType = ((NSNumber*) [data objectForKey:@"new_room_type"]).intValue;
int subEventType = ((NSNumber*) [data objectForKey:@"sub_event_type"]).intValue;
}
}
Data details
|
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | result;error_info;new_room_type;subEventType | {"error_info":"","new_room_type":0,"subEventType":0,"result":0} |
The monitoring event of room call quality
The message for quality monitoring event triggered once every two seconds after room entry is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY. The returned parameters include weight, loss, and delay, which represent the following information.
This API is used to monitor the network quality. If the user's network is poor, the business layer will remind the user to switch to a better network through the UI.
|
| | Value range: 1-50. 50 indicates excellent sound quality, 1 indicates very poor (barely usable) sound quality, and 0 represents an initial meaningless value. Generally, if the value is below 30, you can remind users that the network is poor and recommend them to switch the network. |
| | Upstream packet loss rate |
| | |
Getting version number
This API is used to get the SDK version number for analysis.
Function prototype
-(NSString*)GetSDKVersion;
Sample code
[[ITMGContext GetInstance] GetSDKVersion];
Checking mic permission
This API is used to return the mic permission status.
Function prototype
-(ITMG_RECORD_PERMISSION)CheckMicPermission;
Parameter description
|
| | Mic permission is granted. |
| | |
ITMG_PERMISSION_NotDetermined | | No authorization box has been popped up to request the permission. |
| | An error occurred while calling the API. |
Sample code
[[ITMGContext GetInstance] CheckMicPermission];
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
Function prototype
-(void)SetLogLevel:(ITMG_LOG_LEVEL)levelWrite (ITMG_LOG_LEVEL)levelPrint;
Parameter description
|
| | Sets the level of logs to be written. TMG_LOG_LEVEL_NONE indicates not to write. Default value: TMG_LOG_LEVEL_INFO |
| | Sets the level of logs to be printed. TMG_LOG_LEVEL_NONE indicates not to print. Default value: TMG_LOG_LEVEL_ERROR |
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
[[ITMGContext GetInstance] SetLogLevel:TMG_LOG_LEVEL_INFO TMG_LOG_LEVEL_INFO];
Setting the log printing path
This API is used to set the log printing path, which is /Users/username/Library/Containers/xxx.xxx.xxx/Data/Documents by default.
Function prototype
-(void)SetLogPath:(NSString*)logDir;
Sample code
[[ITMGContext GetInstance] SetLogPath:Path];
Getting the diagnostic messages
This API is used to get information on the quality of real-time audio/video calls, which is mainly used to view real-time call quality and troubleshoot and can be ignored on the business side.
Function prototype
-(NSString*)GetQualityTips;
Sample code
[[[ITMGContext GetInstance]GetRoom ] GetQualityTips];
Callback Messages
Message list
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | Indicates that a member enters an audio room. |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | Indicates that a member exits an audio room. |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | Indicates that a room is disconnected for network or other reasons. |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | Indicates a room type change event. |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | Indicates that the room members are updated. |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | Indicates the room quality information. |
Data list
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | | {"error_info":"waiting timeout, please check your network","result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | result; error_info; sub_event_type; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_NEW_DEVICE | | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_LOST_DEVICE | | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_NEW_DEVICE | | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_LOST_DEVICE | | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | | {"event_id":1,"user_list":["0"]} |
ITMG_MAIN_EVENT_TYPE_NUMBER_OF_USERS_UPDATE | AllUser; AccUser; ProxyUser | {"AllUser":3,"AccUser":2,"ProxyUser":1} |
ITMG_MAIN_EVENT_TYPE_NUMBER_OF_AUDIOSTREAMS_UPDATE | | |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | | {"weight":5,"loss":0.1,"delay":1} |
This document describes how to access and debug GME client APIs for the voice chat feature for Windows.
Key Considerations for Using GME
GME provides the real-time voice, voice message, and speech-to-text conversion services, which all depend on core APIs such as Init and Poll.
Key notes
You have created a GME application and obtained the AppID and Key of the SDK as instructed in Activating Services. You have activated the real-time voice, voice message, and speech-to-text services of GME as instructed in Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, QAVError.OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Connecting to the SDK
Directions
Key processes involved in SDK connection are as follows:
C++ classes
|
| |
| |
| |
| |
| |
| Sound effect and accompaniment APIs |
| Voice message and speech-to-text conversion APIs |
Key APIs
You need to import the header file tmg_sdk.h first before you can access GME. The classes in the header file inherit ITMGDelegate for message delivery and callback.
Sample code
#include "auth_buffer.h"
#include "tmg_sdk.h"
#include "AdvanceHeaders/tmg_sdk_adv.h"
#include <vector>
Callback
Setting callback sample
// When initializing the SDK
m_pTmgContext = ITMGContextGetInstance();
m_pTmgContext->SetTMGDelegate(this);
// In the destructor
CTMGSDK_For_AudioDlg::~CTMGSDK_For_AudioDlg()
{
ITMGContextGetInstance()->SetTMGDelegate(NULL);
}
Message delivery
The API class uses the Delegate method to send callback notifications to the application. ITMG_MAIN_EVENT_TYPE indicates the message type. The data on Windows is in json string format. For the key-value pairs, please see the relevant documentation.
// Declaration in the header file
virtual void OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data);
// Sample code
void CTMGSDK_For_AudioDlg::OnEvent(ITMG_MAIN_EVENT_TYPE eventType, const char* data)
{
switch(eventType)
{
case ITMG_MAIN_EVENT_TYPE_XXXX_XXXX:
{
// Process the callback
}
break;
}
}
Getting singleton
The GME SDK is provided in the form of a singleton, all calls begin with ITMGContext, and callbacks are passed to the application through ITMGDelegate, which should be configured first.
Sample code
ITMGContext* m_pTmgContext;
m_pTmgContext->Init(AppID, OpenID);
You need to initialize the SDK through the Init API before you can use the real-time voice, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
ITMGContext virtual int Init(const char* sdkAppId, const char* openId)
|
| | |
| | openID can only be in Int64 type, which is passed in after being converted to a const char*. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
Returned values
|
| Initialized SDK successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Checks whether the SDK file is complete. It is recommended to delete it and then import the SDK again. |
Notes on 7015 error code
The 7015 error code is judged by md5. If this error is reported during integration, please check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Due to the third-party reinforcement, Unity packaging mechanism and other factors, the md5 of the library file will be affected, resulting in misjudgment. Please ignore this error in the logic for official release, and try to avoid displaying it in the UI.
Sample code
#define SDKAPPID3RD "14000xxxxx"
cosnt char* openId="10001";
ITMGContext* context = ITMGContextGetInstance();
context->Init(SDKAPPID3RD, openId);
Triggering event callback
Event callbacks can be triggered by periodically calling the Poll API in update. Poll is the message pump of GME, and the Poll API should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run exceptionally.\nYou can refer to the EnginePollHelper.cpp file in the demo.
Calling the `Poll` API periodically
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
class ITMGContext {
protected:
virtual ~ITMGContext() {}
public:
virtual void Poll()= 0;
}
Sample code
// Declaration in the header file
// Code implementation
void TMGTestScene::update(float delta)
{
ITMGContextGetInstance()->Poll();
}
Pausing the system
When a Pause event occurs in the system, the engine should also be notified for pause. If you do not need the background to play back the audio in the room, please call Pause API to pause the GME service.
API prototype
Resuming the system
When a Resume event occurs in the system, the engine should also be notified for resumption. The Resume API only supports resuming voice chat.
API prototype
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Chat Room APIs
You should initialize and call the SDK to enter a room before voice chat can start.
|
| Calculates the local authentication key |
| |
| |
| Determines whether room entry is successful |
| Switches the room quickly |
Local authentication key calculation
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, please use the backend deployment key as detailed in Authentication Key. API prototype
int QAVSDK_AuthBuffer_GenAuthBuffer(unsigned int dwSdkAppID, const char* strRoomID, const char* strOpenID,
const char* strKey, unsigned char* strAuthBuffer, unsigned int bufferLength);
|
| | AppId from the Tencent Cloud console
|
| | Room ID, which can contain up to 127 characters. |
| | User ID, which is the same as openID during initialization. |
| | |
| | |
| | Length of the authbuff passed in. 500 is recommended. |
Sample code
unsigned int bufferLen = 512;
unsigned char retAuthBuff[512] = {0};
QAVSDK_AuthBuffer_GenAuthBuffer(atoi(SDKAPPID3RD), roomId, "10001", AUTHKEY,retAuthBuff,bufferLen);
Entering a room
This API is used to enter a room with the generated authentication information. The mic and speaker are not enabled by default after room entry.
Note:
If the room entry callback result is 0, the room entry is successful. If 0 is returned from the EnterRoom API, it doesn't necessarily mean that the room entry is successful.
The audio type of the room is determined by the first user entering the room. After that, if a member in the room changes the room type, it will take effect for all members there. For example, if the first user entering the room uses the smooth sound quality, and the second user entering the room uses the HD sound quality, the room audio type of the second user will change to the smooth sound quality. Only after a member in the room calls the ChangeRoomType API will the audio type of the room be changed.
API prototype
ITMGContext virtual int EnterRoom(const char* roomID, ITMG_ROOM_TYPE roomType, const char* authBuff, int buffLen)
|
| | Room ID, which can contain up to 127 characters. |
| | Room type. We recommend you select ITMG_ROOM_TYPE_FLUENCY for games. For more information on room audio types, see Sound Quality. |
| | |
| | Authentication key length |
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->EnterRoom(roomID, ITMG_ROOM_TYPE_FLUENCY, (char*)retAuthBuff,bufferLen);
Callback for room entry
After the user enters the room, the message ITMG_MAIN_EVENT_TYPE_ENTER_ROOM will be sent and identified in the OnEvent function for callback and processing. A successful callback means that the room entry is successful, and the billing starts.
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
ListMicDevices();
ListSpeakerDevices();
std::string strText = "EnterRoom complete: ret=";
strText += data;
m_EditMonitor.SetWindowText(MByteToWChar(strText).c_str());
}
}
}
Data details
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | | {"error_info":"waiting timeout, please check your network","result":0} |
If the network is disconnected, there will be a disconnected callback prompt ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT. At this time, the SDK will automatically reconnect, and the callback is ITMG_MAIN_EVENT_TYPE_RECONNECT_START. When the reconnection is successful, there will be a callback ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS.
Error codes
|
| Authentication failed: The AppID does not exist or is incorrect. An error occurred while authenticating the authbuff. Authentication expired. The OpenId does not meet the specification. |
| |
| The user was already in the process of entering a room but repeated this operation. It is recommended not to call the room entering API until the room entry callback is returned. |
| The user was already in the room and called the room entering API again. |
| Make sure that the SDK is initialized, OpenId complies with the rules, the APIs are called in the same thread, and the Poll API is called normally. |
Exiting a room
This API is used to exit the current room. It is an async API. The returned value AV_OK indicates a successful async delivery. If there is a scenario in the application where room entry is performed immediately after room exit, you don't need to wait for the RoomExitComplete callback notification from the ExitRoom API; instead, you can directly call the EnterRoom API.
API prototype
ITMGContext virtual int ExitRoom()
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->ExitRoom();
Callback for room exit
After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM.
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_EXIT_ROOM:
{
// Process
break;
}
}
}
Data details
|
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | | {"error_info":"","result":0} |
Determining whether user has entered room
This API is used to determine whether the user has entered a room. A value in bool type will be returned. Do not call this API during room entry.
API prototype
ITMGContext virtual bool IsRoomEntered()
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->IsRoomEntered();
Switching room
User can call this API to quickly switch the voice chat room after entering the room. After the room is switched, the device is not reset, that is, if the microphone is already enabled in this room, the microphone will keep enabled after the room is switched.\nThe callback for quickly switching rooms is ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM, and the fields are error_info and result.
API prototype
ITMGContext virtual int SwitchRoom(const char* targetRoomID, const char* authBuff, int buffLen);
Type descriptions
|
| | |
| | Generates a new authentication key with the ID of the room to enter |
| | Authentication key length |
Room Status Maintenance
APIs in this section are used to display speaking members and members entering or exiting the room and mute a member in the room at the business layer.
|
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | The member status changed |
| Mutes a member in the room |
| |
Notification events of member room entry and speaking status
This API is used to obtain the user speaking in the room and display it in the UI, and to send a notification when someone entering or exiting the room.\nNotification for this event will be sent only when the status changes. To get member status in real time, cache the notification when it is received at a higher layer. The event message is ITMG_MAIN_EVNET_TYPE_USER_UPDATE, where the data contains event_id and user_list. The event message will be identified in the OnEvent function.\nNotifications for audio events are subject to a threshold, and a notification will be sent only when this threshold is exceeded. The notification "A member has stopped sending audio packets" will be sent if no audio packets are received in more than two seconds.
|
| Return the openid of the member entering the room. | |
| Return the openid of the member exiting the room. | |
ITMG_EVENT_ID_USER_HAS_AUDIO | Return the openid of the member sending audio packets in the room. This event can be used to determine whether a user is speaking and display the voiceprint effect. | |
ITMG_EVENT_ID_USER_NO_AUDIO | Return the openid of the member stopping sending audio packets in the room. | |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_USER_UPDATE:
{
// Process
// Parse the parameter to get `eventID` and `user_list`
switch (eventID)
{
case ITMG_EVENT_ID_USER_ENTER:
// A member enters the room
break;
case ITMG_EVENT_ID_USER_EXIT:
// A member exits the room
break;
case ITMG_EVENT_ID_USER_HAS_AUDIO:
// A member sends audio packets
break;
case ITMG_EVENT_ID_USER_NO_AUDIO:
// A member stops sending audio packets
break;
default:
break;
}
break;
}
}
}
Muting a member in the room
This API is used to add an ID to the audio data blocklist. This operation blocks audio from someone and only applies to the local device without affecting other devices. The returned value 0 indicates that the call is successful. Assume that users A, B, and C are all speaking using their mic in a room:
If A blocks C, A can only hear B;
If B blocks neither A nor C, B can hear both of them;
If C blocks neither A nor B, C can hear both of them.
This API is suitable for scenarios where a user is muted in a room.
API prototype
ITMGContext ITMGAudioCtrl int AddAudioBlackList(const char* openId)
|
| | openid of the user to be blocked
|
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->AddAudioBlackList(openId);
Unmuting
This API is used to remove an ID from the audio data blacklist. A returned value of 0 indicates the call is successful.
API prototype
ITMGContext ITMGAudioCtrl int RemoveAudioBlackList(const char* openId)
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->RemoveAudioBlackList(openId);
Voice Chat Capturing APIs
The voice chat APIs can only be called after SDK initialization and room entry.
When the user clicks the button of enabling/disabling the mic or speaker on the UI, we recommend you call the EnableMic or EnableSpeaker API.
To enable the user to press the mic button on the UI to speak and release it to stop speaking, we recommend you call EnableAudioCaptureDevice once during room entry and call EnableAudioSend to enable the user to speak while pressing the button.
|
| |
| |
| Enables/Disables the capturing device |
IsAudioCaptureDeviceEnabled | Gets the capturing device status |
| Enables/Disables audio upstreaming |
| Gets the audio upstreaming status |
| Gets the real-time mic volume level |
| Gets real-time audio upstreaming volume |
| Sets the mic volume level |
| Gets the mic volume level |
Enabling or disabling mic
This API is used to enable/disable the mic. The mic and speaker are not enabled by default after room entry. EnableMic = EnableAudioCaptureDevice + EnableAudioSend
API prototype
ITMGAudioCtrl virtual int EnableMic(bool bEnabled)
|
| | To enable the mic, set this parameter to true, otherwise, set it to false. |
Sample code
// Enable mic
ITMGContextGetInstance()->GetAudioCtrl()->EnableMic(true);
Getting the mic status
This API is used to get the mic status. The returned value 0 indicates that the mic is off, while 1 is on.
API prototype
ITMGAudioCtrl virtual int GetMicState()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicState();
Enabling or disabling capturing device
This API is used to enable/disable a capturing device. The device is not enabled by default after room entry.
This API can only be called after room entry. The device will be disabled automatically after room exit.
Operations such as permission application and volume type adjustment will generally be performed when a capturing device is enabled on a mobile device.
API prototype
ITMGAudioCtrl virtual int EnableAudioCaptureDevice(bool enable)
|
| | To enable the capturing device, set this parameter to true, otherwise, set it to false. |
Sample code
// Enable capturing device
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioCaptureDevice(true);
Getting the capturing device status
This API is used to get the status of a capturing device.
API prototype
ITMGContext virtual bool IsAudioCaptureDeviceEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioCaptureDeviceEnabled();
Enabling or disabling audio upstreaming
This API is used to enable/disable audio upstreaming. If a capturing device is already enabled, it will send captured audio data; otherwise, it will remain mute. For more information on how to enable/disable the capturing device, please see the EnableAudioCaptureDevice API.
API prototype
ITMGContext virtual int EnableAudioSend(bool bEnable)
|
| | To enable audio upstreaming, set this parameter to true; otherwise, set it to false. |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioSend(true);
Getting audio upstreaming status
This API is used to get the status of audio upstreaming.
API prototype
ITMGContext virtual bool IsAudioSendEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioSendEnabled();
Getting the real-time mic volume
This API is used to get the real-time mic volume. An int-type value in the range of 0-100 will be returned. It is recommended to call this API once every 20 ms.
API prototype
ITMGAudioCtrl virtual int GetMicLevel()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicLevel();
Getting the real-time audio upstreaming volume
This API is used to get the local real-time audio upstreaming volume. An int-type value in the range of 0-100 will be returned.
API prototype
ITMGAudioCtrl virtual int GetSendStreamLevel()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSendStreamLevel();
Setting the mic software volume
This API is used to set the mic volume level. The corresponding parameter is volume, which is equivalent to attenuating or gaining the captured sound.
API prototype
ITMGAudioCtrl virtual int SetMicVolume(int vol)
|
| | Value range: 0–200. Default value: 100. 0 indicates that the audio is mute, while 100 indicates that the volume level remains unchanged. |
Sample code
int micVol = (int)(value * 100);
ITMGContextGetInstance()->GetAudioCtrl()->SetMicVolume(vol);
Getting the mic software volume
This API is used to obtain the microphone volume. An "int" value is returned. Value 101 represents API SetMicVolume has not been called.
API prototype
ITMGAudioCtrl virtual int GetMicVolume()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicVolume();
Voice Chat Playback APIs
|
| Enables/Disables the speaker |
| |
| Enables/Disables the playback device |
| Gets playback device status |
| Enables/Disables audio downstreaming |
| Gets the audio downstreaming status |
| Gets the real-time speaker volume level |
| Gets the real-time downstreaming audio volume levels of other members in the room |
| Sets the speaker volume level |
| Gets the speaker volume level |
Enabling or disabling speaker
This API is used to enable/disable the speaker. EnableSpeaker = EnableAudioPlayDevice + EnableAudioRecv
API prototype
ITMGAudioCtrl virtual int EnableSpeaker(bool enable)
|
| | To disable the speaker, set this parameter to false; otherwise, set it to true. |
Sample code
// Enable the speaker
ITMGContextGetInstance()->GetAudioCtrl()->EnableSpeaker(true);
Getting the speaker status
This API is used to get the speaker status. 0 indicates that the speaker is off, and 1 is on.
API prototype
ITMGAudioCtrl virtual int GetSpeakerState()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerState();
Enabling or disabling playback device
This API is used to enable/disable a playback device.
API prototype
ITMGAudioCtrl virtual int EnableAudioPlayDevice(bool enable)
|
| | To disable the playback device, set this parameter to false; otherwise, set it to true. |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioPlayDevice(true);
Getting the playback device status
This API is used to get the status of a playback device.
API prototype
ITMGAudioCtrl virtual bool IsAudioPlayDeviceEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioPlayDeviceEnabled();
Enabling or disabling audio downstreaming
This API is used to enable/disable audio downstreaming. If a playback device is already enabled, it will play back audio data from other members in the room; otherwise, it will remain mute. For more information on how to enable/disable the playback device, please see the EnableAudioPlayDevice API.
API prototype
ITMGAudioCtrl virtual int EnableAudioRecv(bool enable)
|
| | To enable audio downstreaming, set this parameter to true; otherwise, set it to false. |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioRecv(true);
Getting audio downstreaming status
This API is used to get the status of audio downstreaming.
API prototype
ITMGAudioCtrl virtual bool IsAudioRecvEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioRecvEnabled();
Getting the real-time speaker volume
This API is used to get the real-time speaker volume. An int-type value will be returned to indicate the volume. It is recommended to call this API once every 20 ms.
API prototype
ITMGAudioCtrl virtual int GetSpeakerLevel()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerLevel();
Getting the real-time downstreaming audio levels of other members in room
This API is used to get the real-time audio downstreaming volume of other members in the room. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGAudioCtrl virtual int GetRecvStreamLevel(const char* openId)
|
| | openId of other members in the room
|
Sample code
iter->second.level = ITMGContextGetInstance()->GetAudioCtrl()->GetRecvStreamLevel(iter->second.openid.c_str());
Dynamically setting the volume of a member of the room
This API is used to set the volume of a member in the room. It takes effect only on the local.
API prototype
ITMGAudioCtrl virtual int SetSpeakerVolumeByOpenID(const char* openId, int vol) = 0;
|
| | OpenID of the user whose volume level needs to be set
|
| | Percentage. Recommended value range: 0–200. Default value: 100. |
Getting volume percentage
This API is used to get the volume level set by SetSpeakerVolumeByOpenID.
API prototype
ITMGAudioCtrl virtual int GetSpeakerVolumeByOpenID(const char* openId) = 0;
|
| | OpenID of the user whose volume level needs to be set
|
Returned values
API returns volume percentage set by OpenID, where 100 is by default.
Setting the speaker volume
This API is used to set the speaker volume.
API prototype
ITMGAudioCtrl virtual int SetSpeakerVolume(int vol)
|
| | Volume level. Value range: 0–200. Default value: 100. 0 indicates that the audio is mute, while 100 indicates that the volume level remains unchanged. |
Sample code
int vol = 100;
ITMGContextGetInstance()->GetAudioCtrl()->SetSpeakerVolume(vol);
Getting the speaker volume
This API is used to get the speaker volume. An int-type value will be returned to indicate the volume. 101 indicates that the SetSpeakerVolume API has not been called.\n"Level" indicates the real-time volume, and "Volume" the speaker volume. The final volume = Level * Volume%. For example, if the "Level" is 100 and "Volume" is 60, the final volume is "60".
API prototype
ITMGAudioCtrl virtual int GetSpeakerVolume()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerVolume();
Advanced APIs
Enabling in-ear monitoring
This API is used to enable in-ear monitoring. You need to call EnableLoopBack+EnableSpeaker before you can hear your own voice.
API prototype
ITMGAudioCtrl virtual int EnableLoopBack(bool enable)
|
| | Specifies whether to enable |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableLoopBack(true);
Getting user's room audio type
This API is used to get a user's room audio type. The returned value is the room audio type. Value 0 indicates that an error occurred while getting the user's room audio type. For room audio types, please see the EnterRoom API.
API prototype
class ITMGRoom {
public:
virtual ~ITMGRoom() {} ;
virtual int GetRoomType() = 0;
};
Sample code
ITMGContext* context = ITMGContextGetInstance();
ITMGContextGetInstance()->GetRoom()->GetRoomType();
Getting the room ID
This API is used to get the voice chat room ID and can be called only after a successful room entry.
API prototype
ITMGRoom virtual int GetRoomID(char* pBuffer, int nLength) = 0;
|
| | It is used to receive the returned roomid. |
| | pBuffer length. Value range: 128–256.
|
Modifying user's room audio type
This API is used to modify a user's room audio type. For the result, please see the callback event. The event type is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE. The audio type of the room is determined by the first user to enter the room. After that, if a member in the room changes the room type, it will take effect for all members there.
API prototype
IITMGContext TMGRoom public int ChangeRoomType((ITMG_ROOM_TYPE roomType)
|
| | Room type to be switched to the target type. For room audio types, please see the EnterRoom API. |
Sample code
ITMGContext* context = ITMGContextGetInstance();
ITMGContextGetInstance()->GetRoom()->ChangeRoomType(ITMG_ROOM_TYPE_FLUENCY);
Data details
|
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | result; error_info; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
Callback for modifying the room type
After the room type is set, the event message ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE will be returned in the callback. The returned parameters include result, error_info, and new_room_type. The new_room_type represents the following information. The event message will be identified in the OnEvent function.
|
ITMG_ROOM_CHANGE_EVENT_ENTERROOM | | The existing audio type is inconsistent with and changed to that of the entered room. |
ITMG_ROOM_CHANGE_EVENT_START | | A user is already in the room and the audio type starts changing (e.g., calling the ChangeRoomType API to change the audio type). |
ITMG_ROOM_CHANGE_EVENT_COMPLETE | | A user is already in the room, and the audio type has been changed. |
ITMG_ROOM_CHANGE_EVENT_REQUEST | | A room member calls the ChangeRoomType API to request a change of room audio type. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
if (ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE == type)
{
// Process room type events
}
}
The monitoring event of room call quality
This is the quality monitoring event, which is triggered once every two seconds after room entry, and its message is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY. The returned parameters include weight, loss, and delay, which are as detailed below:
|
| | Value range: 1-50. 50 indicates excellent sound quality, 1 indicates very poor (barely usable) sound quality, and 0 represents an initial meaningless value. Generally, if the value is below 30, the business layer will remind users that the network is poor and recommend them to switch the network. |
| | Upstream packet loss rate |
| | |
Getting version number
This API is used to get the SDK version number for analysis.
API prototype
ITMGContext virtual const char* GetSDKVersion()
Sample code
ITMGContextGetInstance()->GetSDKVersion();
Checking mic status
Function prototype
ITMGContext virtual ITMG_CHECK_MIC_STATUS CheckMic() = 0;
Returned value handling
|
ITMG_CHECK_MIC_STATUS_AVAILABLE = 0 | | |
ITMG_CHECK_MIC_STATUS_NO_GRANTED = 2 | Access not obtained/denied | The access permission needs to be obtained before the mic is enabled |
ITMG_CHECK_MIC_STATUS_INVALID_MIC = 3 | | Generally, this error will be reported on PCs when no mics are available. Prompt the user to insert a headset or mic |
ITMG_CHECK_MIC_STATUS_NOT_INIT = 5 | | Call EnableMic after Init |
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
ITMGContext int SetLogLevel(ITMG_LOG_LEVEL levelWrite, ITMG_LOG_LEVEL levelPrint)
Parameter description
|
| | Sets the level of logs to be written. TMG_LOG_LEVEL_NONE indicates not to write. Default value: TMG_LOG_LEVEL_INFO |
| | Sets the level of logs to be printed. TMG_LOG_LEVEL_NONE indicates not to print. Default value: TMG_LOG_LEVEL_ERROR |
ITMG_LOG_LEVEL is as detailed below:
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
ITMGContextGetInstance()->SetLogLevel(TMG_LOG_LEVEL_INFO,TMG_LOG_LEVEL_INFO);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
|
| %appdata%\\Tencent\\GME\\ProcessName |
| Application/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/Documents |
| /sdcard/Android/data/xxx.xxx.xxx/files |
| /Users/username/Library/Containers/xxx.xxx.xxx/Data/Documents |
API prototype
ITMGContext virtual int SetLogPath(const char* logDir)
Sample code
cosnt char* logDir = ""// Set a path by yourself
ITMGContext* context = ITMGContextGetInstance();
context->SetLogPath(logDir);
Getting the printed log path
This API is used to get the log path. Its returned value is a string of const char* type.
API prototype
ITMGContext virtual const char* GetLogPath() = 0;
Getting the diagnostic messages
This API is used to get information on the quality of real-time audio/video calls, which is mainly used to view real-time call quality and troubleshoot and can be ignored on the business side.
API prototype
ITMGRoom virtual const char* GetQualityTips()
Sample code
ITMGContextGetInstance()->GetRoom()->GetQualityTips();
Callback Messages
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | A member entered the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | A member exited the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | The room was disconnected for network or other reasons | result; error_info | {"error_info":"waiting timeout, please check your network","result":0} |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | The room members were updated | user_list; event_id | {"event_id":1,"user_list":["0"]} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_START | Room reconnection started | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS | Room reconnection succeeded | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM | The room was quickly switched | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | The room status changed | result; error_info; sub_event_type; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_START | Cross-room mic connect started | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_STOP | Cross-room mic connect stopped | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_DEFAULT_DEVICE_CHANGED | The default speaker was changed | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_NEW_DEVICE | A speaker was added | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_LOST_DEVICE | A speaker was lost | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_NEW_DEVICE | A mic was added | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"mic (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_LOST_DEVICE | A mic was lost | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_DEFAULT_DEVICE_CHANGED | The default mic was changed | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | Room quality message | weight; loss; delay | {"weight":5,"loss":0.1,"delay":1} |
ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE | Recording of a voice message was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE | Upload of a voice message was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE | Download of a voice message was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE | Playback of a voice message was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE | Fast recording-to-text conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE | Streaming speech-to-text conversion was completed | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING | A voice message is being converted into text in a streaming manner | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TEXT2SPEECH_COMPLETE | Text-to-speech conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TRANSLATE_TEXT_COMPLETE | Text translation was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
This document describes how to integrate with and debug GME client APIs for the voice messaging and speech-to-text services for Windows.
Key Considerations for Using GME
GME provides the real-time voice and voice messaging and speech-to-text services, which all depend on core APIs such as Init and Poll.
Notes
You have created a GME application and obtained the SDK AppID and key. For more information, see Activating Services. You have activated GME real-time voice and voice messaging and speech-to-text services. For more information, see Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, QAVError.OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Note:
There is a default call rate limit for speech-to-text APIs. For more information on how calls are billed within the limit, see Purchase Guide. If you want to increase the limit or learn more about how excessive calls are billed, submit a ticket. Non-streaming speech-to-text API SpeechToText(): There can be up to 10 concurrent requests per account.
Streaming speech-to-text API StartRecordingWithStreamingRecognition(): There can be up to 50 concurrent requests per account.
Real-time streaming speech-to-text API StartRealTimeASR(): There can be up to 50 concurrent requests per account.
Integrating the SDK
Directions
Key processes involved in SDK integration are as follows:
C++ classes
|
| |
| Voice messaging and speech-to-text APIs |
Key APIs
#include "auth_buffer.h"
#include "tmg_sdk.h"
#include "AdvanceHeaders/tmg_sdk_adv.h"
#include <vector>
Callback
Setting callback sample
// When initializing the SDK
m_pTmgContext = ITMGContextGetInstance();
m_pTmgContext->SetTMGDelegate(this);
// In the destructor
CTMGSDK_For_AudioDlg::~CTMGSDK_For_AudioDlg()
{
ITMGContextGetInstance()->SetTMGDelegate(NULL);
}
Message delivery
The API class uses the Delegate method to send callback notifications to the application. ITMG_MAIN_EVENT_TYPE indicates the message type. The data on Windows is in json string format. For the key-value pairs, please see the relevant documentation.
// Declaration in the header file
virtual void OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data);
// Sample code
void CTMGSDK_For_AudioDlg::OnEvent(ITMG_MAIN_EVENT_TYPE eventType, const char* data)
{
switch(eventType)
{
case ITMG_MAIN_EVENT_TYPE_XXXX_XXXX:
{
// Process the callback
}
break;
}
}
Getting singleton
The GME SDK is provided in the form of a singleton, all calls begin with ITMGContext, and callbacks are passed to the application through ITMGDelegate, which should be configured first.
Sample code
ITMGContext* m_pTmgContext;
m_pTmgContext->Init(AppID, OpenID);
Initializing the SDK
You need to initialize the SDK through the Init API before you can use the real-time voice, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
ITMGContext virtual int Init(const char* sdkAppId, const char* openId)
|
| | |
| | openID can only be in Int64 type, which is passed in after being converted to a const char*. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
Returned values
|
| Initialized SDK successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Checks whether the SDK file is complete. It is recommended to delete it and then import the SDK again. |
Notes on 7015 error code
The 7015 error code is judged by md5. If this error is reported during integration, please check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Due to the third-party reinforcement, Unity packaging mechanism and other factors, the md5 of the library file will be affected, resulting in misjudgment. Please ignore this error in the logic for official release, and try to avoid displaying it in the UI.
Sample code
#define SDKAPPID3RD "14000xxxxxx"
cosnt char* openId="10001";
ITMGContext* context = ITMGContextGetInstance();
context->Init(SDKAPPID3RD, openId);
Triggering event callback
Event callbacks can be triggered by periodically calling the Poll API in update. Poll is the message pump of GME, and the Poll API should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run exceptionally.\nYou can refer to the EnginePollHelper.cpp file in the demo.
Note:
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
class ITMGContext {
protected:
virtual ~ITMGContext() {}
public:
virtual void Poll()= 0;
}
Sample code
void TMGTestScene::update(float delta)
{
ITMGContextGetInstance()->Poll();
}
Pausing the system
When a Pause event occurs in the system, the engine should also be notified for pause. If you do not need the background to play back the audio in the room, please call Pause API to pause the GME service.
API prototype
Resuming the system
When a Resume event occurs in the system, the engine should also be notified for resumption. The Resume API only supports resuming voice chat.
API prototype
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Messaging and Speech-to-Text Services
Note:
The speech-to-text service consists of fast recording-to-text conversion and streaming speech-to-text conversion.
You do not need to enter a voice chat room when using the voice message service.
The maximum recording duration of a voice message is 58 seconds by default, and the minimum recording duration cannot be less than 1 second. If you want to customize the recording duration, for example, to modify the maximum recording duration to 10 seconds, call the SetMaxMessageLength API to set it after initialization.
Flowchart for using the speech-to-text service
|
| Generates the local authentication key |
| Initializes authentication |
| Specifies the maximum length of voice message |
Generating the local authentication key
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, please use the backend deployment key as detailed in Authentication Key. API prototype
int QAVSDK_AuthBuffer_GenAuthBuffer(unsigned int dwSdkAppID, const char* strRoomID, const char* strOpenID,
const char* strKey, unsigned char* strAuthBuffer, unsigned int bufferLength);
|
| | AppId from the Tencent Cloud console.
|
| | Enter null or an empty string |
| | User ID, which is the same as openID during initialization. |
| | Permission key from the Tencent Cloud console. |
| | |
| | Length of the authbuff passed in. 500 is recommended. |
Application authentication
After the authentication information is generated, the authentication is assigned to the SDK.
API prototype
ITMGPTT virtual int ApplyPTTAuthbuffer(const char* authBuffer, int authBufferLen)
Sample code
ITMGContextGetInstance()->GetPTT()->ApplyPTTAuthbuffer(authBuffer,authBufferLen);
Specifying the maximum duration of voice message
This API is used to specify the maximum duration of a voice message, which can be up to 58 seconds.
API prototype
ITMGPTT virtual int SetMaxMessageLength(int msTime)
|
| | Audio duration in ms. Value range: 1000 < msTime <= 58000 |
Sample code
int msTime = 10000;
ITMGContextGetInstance()->GetPTT()->SetMaxMessageLength(msTime);
Streaming Speech Recognition
Voice message and speech-to-text APIs
|
StartRecordingWithStreamingRecognition | Starts streaming recording |
| |
Starting streaming speech recognition
This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. It can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation. To stop recording, call Stop recording. API prototype
ITMGPTT virtual int StartRecordingWithStreamingRecognition(const char* filePath)
ITMGPTT virtual int StartRecordingWithStreamingRecognition(const char* filePath,const char* translateLanguage,const char* translateLanguage)
|
| | Path of stored audio file |
| | |
| | |
Sample code
ITMGContextGetInstance()->GetPTT()->StartRecordingWithStreamingRecognition(filePath,"cmn-Hans-CN","cmn-Hans-CN");
Note:
Translation incurs additional fees. For more information, see Purchase Guide. Callback for streaming speech recognition
After streaming speech recognition is started, you need to listen on callback messages in the OnEvent notification, which is as detailed below:
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING returns the recognized text in real-time during the recording, which is equivalent to returning the recognized text while speaking.
The event message will be identified in the OnEvent notification based on the actual needs. The passed parameters include the following four messages.
|
| Return code indicating whether streaming speech recognition is successful |
| Text converted from speech |
| Local path of stored recording file |
| Backend URL address of recording file, which will be retained for 90 days |
Note:
The file_id is empty when the 'ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRecognition_IS_RUNNING' message is listened.
Error codes
|
| Streaming speech-to-text conversion failed, but recording succeeded. | Call the UploadRecordedFile API to upload the recording file and then call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed, but recording and upload succeeded. | The message returned contains a backend URL after successful upload. Call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed. | During streaming recording, wait for the execution result of the streaming recording API to return. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
If the error code 4098 is reported, see FAQs for solutions. Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE:
{
HandleSTREAM2TEXTComplete(data,true);
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING:
{
HandleSTREAM2TEXTComplete(data, false);
break;
}
}
}
void CTMGSDK_For_AudioDlg::HandleSTREAM2TEXTComplete(const char* data, bool isComplete)
{
std::string strText = "STREAM2TEXT: ret=";
strText += data;
m_EditMonitor.SetWindowText(MByteToWChar(strText).c_str());
Json::Reader reader;
Json::Value root;
bool parseRet = reader.parse(data, root);
if (!parseRet) {
::SetWindowText(m_EditInfo.GetSafeHwnd(),MByteToWChar(std::string("parse result Json error")).c_str());
}
else
{
if (isComplete) {
::SetWindowText(m_EditUpload.GetSafeHwnd(), MByteToWChar(root["file_id"].asString()).c_str());
}
else {
std::string isruning = "STREAMINGRECOGNITION_IS_RUNNING";
::SetWindowText(m_EditUpload.GetSafeHwnd(), MByteToWChar(isruning).c_str());
}
}
}
Voice Message Recording
The recording process is as follows: start recording > stop recording > return recording callback > start the next recording.
Voice message and speech-to-text APIs
Starting recording
This API is used to start recording.
API prototype
ITMGPTT virtual int StartRecording(const char* fileDir)
|
| | Path of stored audio file |
Sample code
char buffer[256]={0};
snprintf(buffer, sizeof(buffer), "%sunreal_ptt_local.file", getFilePath().c_str());
ITMGContextGetInstance()->GetPTT()->StartRecording(buffer);
Stopping recording
This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.
API prototype
ITMGPTT virtual int StopRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->StopRecording();
Callback for recording start
The recording start result will be returned through the callback.
To stop recording, call StopRecording. The callback for recording start will be returned after the recording is stopped.
|
| | 0: recording is completed |
| | Path of stored recording file, which must be accessible and cannot be the fileid |
Error codes
|
| | Check whether the API parameters in the code are correct. |
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| Recording is in progress. | Ensure that the SDK recording feature is used at the right time. |
| Audio data is not captured. | Check whether the mic is working properly. |
| An error occurred while accessing the file during recording. | Ensure the existence of the file and the validity of the file path. |
| The mic is not authorized. | Mic permission is required for using the SDK. To add the permission, please see the SDK project configuration document for the corresponding engine or platform. |
| The recording duration is too short. | The recording duration should be in ms and longer than 1,000 ms. |
| No recording operation is started. | Check whether the recording starting API has been called. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE:
{
// Process
break;
}
}
}
Pausing recording
This API is used to pause recording. If you want to resume recording, please call the ResumeRecording API.
API prototype
ITMGPTT virtual int PauseRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->PauseRecording();
Resuming recording
This API is used to resume recording.
API prototype
ITMGPTT virtual int ResumeRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->ResumeRecording();
Canceling recording
This API is used to cancel recording. There is no callback after cancellation.
API prototype
ITMGPTT virtual int CancelRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->CancelRecording();
Voice Message Upload, Download, and Playback
|
| |
| |
| Plays back the audio file |
| Stops playing back the audio file |
| |
| Gets the audio file duration |
Uploading an audio file
This API is used to upload an audio file.
API prototype
ITMGPTT virtual int UploadRecordedFile(const char* filePath)
|
| | Path of uploaded audio file, which is a local path |
Sample code
ITMGContextGetInstance()->GetPTT()->UploadRecordedFile(filePath);
Callback for audio file upload completion
After the audio file is uploaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
| | 0: recording is completed |
| | Path of stored recording file |
| | |
Error codes
|
| An error occurred while accessing the file during upload. | Ensure the existence of the file and the validity of the file path. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| A network error occurred. | Check whether the device can access the internet. |
| The network failed while getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| The packet returned during the process of getting the upload parameters is empty. | Check whether the authentication is correct and whether the device network can normally access the internet. |
| Failed to decode the packet returned during the process of getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| | Check whether the apply API is called or whether the input parameters are empty. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE:
{
// Process
break;
}
}
}
Downloading the audio file
This API is used to download an audio file.
API prototype
ITMGPTT virtual int DownloadRecordedFile(const char* fileId, const char* filePath)
Sample code
ITMGContextGetInstance()->GetPTT()->DownloadRecordedFile(fileID,filePath);
Callback for audio file download completion
After the audio file is downloaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
| | 0: recording is completed |
| | Path of stored recording file |
| | URL path of file, which will be retained on the server for 90 days |
Error codes
|
| An error occurred while accessing the file during download. | Check whether the file path is valid. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| Network storage system exception. | The server failed to get the audio file. Check whether the API parameter fileid is correct, whether the network is normal, and whether the file exists in COS. |
| Server file system error. | Check whether the device can access the internet and whether the file exists on the server. |
| The HTTP network failed during the process of getting the download parameters. | Check whether the device can access the internet. |
| The packet returned during the process of getting the download parameters is empty. | Check whether the device can access the internet. |
| Failed to decode the packet returned during the process of getting the download parameters. | Check whether the device can access the internet. |
| | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
Playing back audio
This API is used to play back audio.
API prototype
ITMGPTT virtual int PlayRecordedFile(const char* filePath)
ITMGPTT virtual int PlayRecordedFile(const char* filePath, nt voiceType)
Error codes
|
| | Ensure the existence of the file and the validity of the file path. |
Sample code
ITMGContextGetInstance()->GetPTT()->PlayRecordedFile(filePath);
Callback for audio playback
After the audio is played back, the event message ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameter includes result and file_path.
|
| | |
| | Path of stored recording file |
Error codes
|
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| During playback, the client tried to interrupt and play back the next one but failed (which should succeed normally). | Check whether the code logic is correct. |
| | Check whether the API parameters in the code are correct. |
| | An error occurred while initializing the player. This error code is generally caused by failure in decoding, and the error should be located with the aid of logs. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
...
else if (eventType == ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE) {
int32 result = JsonObject->GetIntegerField(TEXT("result"));
FString filepath = JsonObject->GetStringField(TEXT("file_path"));
onPttPlayFileCompleted(result, filepath);
}
}
}
Stopping audio playback
This API is used to stop audio playback. There will be a callback for playback completion when the playback stops.
API prototype
ITMGPTT virtual int StopPlayFile()
Sample code
ITMGContextGetInstance()->GetPTT()->StopPlayFile();
Getting audio file size
This API is used to get the size of an audio file.
API prototype
ITMGPTT virtual int GetFileSize(const char* filePath)
|
| | Path of audio file, which is a local path |
Sample code
ITMGContextGetInstance()->GetPTT()->GetFileSize(filePath);
Getting audio file duration
This API is used to get the duration of an audio file in milliseconds.
API prototype
ITMGPTT virtual int GetVoiceFileDuration(const char* filePath)
|
| | Path of audio file, which is a local path |
Sample code
ITMGContextGetInstance()->GetPTT()->GetVoiceFileDuration(filePath);
Fast Recording-to-Text Conversion
Converting audio file to text
This API is used to convert a specified audio file to text.
API prototype
ITMGPTT virtual void SpeechToText(const char* fileID)
Sample code
ITMGContextGetInstance()->GetPTT()->SpeechToText(fileID);
Translating audio file into text in specified language
This API can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation.
Note:
Translation incurs additional fees. For more information, see Purchase Guide. API prototype
ITMGPTT virtual int SpeechToText(const char* fileID,const char* speechLanguage)
ITMGPTT virtual int SpeechToText(const char* fileID,const char* speechLanguage,const char* translateLanguage)
|
| | The URL of the audio file, which will be retained on the server for 90 days. |
| | |
| | |
Sample code
ITMGContextGetInstance()->GetPTT()->SpeechToText(filePath,"cmn-Hans-CN","cmn-Hans-CN");
Callback for recognition
After the specified audio file is converted to text, the event message ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path and text (recognized text).
|
| | 0: recording is completed |
| | URL of recording file, which will be retained on the server for 90 days |
| | |
Error codes
|
| An internal error occurred. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the device can access the internet. |
| Failed to decode the returned packet. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
| | Check whether authbuffer is correct. |
| Incorrect speech-to-text conversion parameter. | Check whether the API parameter fileid in the code is empty. |
| Speech-to-text translation returned an error. | Error with the backend of voice messaging and speech-to-text feature. Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE:
{
// Process
break;
}
}
}
Voice Message Volume Level APIs
|
| Gets real-time mic volume level |
| Sets recording volume level |
| Gets recording volume level |
| Gets real-time speaker volume |
| Sets playback volume level |
| Gets playback volume level |
Getting the real-time mic volume of voice message
This API is used to get the real-time mic volume. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetMicLevel()
Sample code
ITMGContext.GetInstance(this).GetPTT().GetMicLevel();
Setting the recording volume of voice message
This API is used to set the recording volume of voice message. Value range: 0-200.
API prototype
ITMGPTT virtual int SetMicVolume(int vol)
Sample code
ITMGContextGetInstance()->GetPTT()->SetMicVolume(100);
Getting the recording volume of voice message
This API is used to get the recording volume of voice message. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetMicVolume()
Sample code
ITMGContextGetInstance()->GetPTT()->GetMicVolume();
Getting the real-time speaker volume of voice message
This API is used to get the real-time speaker volume. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetSpeakerLevel()
Sample code
ITMGContextGetInstance()->GetPTT()->GetSpeakerLevel();
Setting the playback volume of voice message
This API is used to set the playback volume of voice message. Value range: 0-200.
API prototype
ITMGPTT virtual int SetSpeakerVolume(int vol)
Sample code
ITMGContextGetInstance()->GetPTT()->SetSpeakerVolume(100);
Getting the playback volume of voice message
This API is used to get the playback volume of voice message. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetSpeakerVolume()
Sample code
ITMGContextGetInstance()->GetPTT()->GetSpeakerVolume();
Advanced APIs
Getting version number
This API is used to get the SDK version number for SDK usage analysis.
API prototype
ITMGContext virtual const char* GetSDKVersion()
Sample code
ITMGContextGetInstance()->GetSDKVersion();
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
ITMGContext int SetLogLevel(ITMG_LOG_LEVEL levelWrite, ITMG_LOG_LEVEL levelPrint)
Parameter description
|
| | Sets the level of logs to be written. TMG_LOG_LEVEL_NONE indicates not to write. Default value: TMG_LOG_LEVEL_INFO |
| | Sets the level of logs to be printed. TMG_LOG_LEVEL_NONE indicates not to print. Default value: TMG_LOG_LEVEL_ERROR |
ITMG_LOG_LEVEL description:
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
ITMGContextGetInstance()->SetLogLevel(TMG_LOG_LEVEL_INFO,TMG_LOG_LEVEL_INFO);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
|
| %appdata%\\Tencent\\GME\\ProcessName |
| Application/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/Documents |
| /sdcard/Android/data/xxx.xxx.xxx/files |
| /Users/username/Library/Containers/xxx.xxx.xxx/Data/Documents |
API prototype
ITMGContext virtual int SetLogPath(const char* logDir)
Sample code
cosnt char* logDir = ""// Set a path by yourself
ITMGContext* context = ITMGContextGetInstance();
context->SetLogPath(logDir);
Callback Messages
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | A member entered the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | A member exited the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | The room was disconnected for network or other reasons | result; error_info | {"error_info":"waiting timeout, please check your network","result":0} |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | Room members were updated | user_list; event_id | {"event_id":1,"user_list":["0"]} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_START | The reconnection to the room started | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS | The reconnection to the room succeeded | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM | The room was quickly switched | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | The room status was changed | result; error_info; sub_event_type; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_START | Cross-room mic connect started | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_STOP | Cross-room mic connect stopped | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_DEFAULT_DEVICE_CHANGED | The default speaker device was changed | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_NEW_DEVICE | A new speaker device was added | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_LOST_DEVICE | A speaker device was lost | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_NEW_DEVICE | A new mic device was added | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_LOST_DEVICE | A mic device was lost | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_DEFAULT_DEVICE_CHANGED | The default mic device was changed | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | The room quality changed | weight; loss; delay | {"weight":5,"loss":0.1,"delay":1} |
ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE | Voice message recording was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE | Voice message upload was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE | Voice message download was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE | Voice message playback was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE | Fast speech-to-text conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE | Streaming speech-to-text conversion was completed | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING | Streaming speech-to-text conversion is in progress | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TEXT2SPEECH_COMPLETE | Text-to-speech conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TRANSLATE_TEXT_COMPLETE | Text translation was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
This document describes how to access and debug GME client APIs for the voice chat feature for Cocos2d.
Key Considerations for Using GME
GME provides the real-time voice, voice message, and speech-to-text services, which all depend on core APIs such as Init and Poll.
Key notes
You have created a GME application and obtained the AppID and Key of the SDK as instructed in Activating Services. You have activated the real-time voice, voice message, and speech-to-text services of GME as instructed in Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, QAVError.OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Connecting to the SDK
Directions
Key processes involved in SDK connection are as follows:
C++ classes
|
| |
| |
| |
| |
| Sound effect and accompaniment APIs |
Key APIs
Preparations
You need to import the header file tmg_sdk.h first before you can access GME. The classes in the header file inherit ITMGDelegate for message delivery and callback.
Sample code
#include "tmg_sdk.h"
class TMGTestScene : public cocos2d::Scene,public ITMGDelegate
{
public:
...
private:
...
｝
Setting a singleton
You need to get ITMGContext first before you can call the EnterRoom function. All calls begin with ITMGContext, which is returned to the application through the ITMGDelegate callback and must be set first.
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->SetTMGDelegate(this);
Message delivery
The API class uses the Delegate method to send callback notifications to the application. ITMG_MAIN_EVENT_TYPE indicates the message type. The data on Windows is in json string format. For the key-value pairs, please see the relevant documentation.
Sample code
// Function implementation:
//TMGTestScene.h:
class TMGTestScene : public cocos2d::Scene,public ITMGDelegate
{
public:
void OnEvent(ITMG_MAIN_EVENT_TYPE eventType, const char* data);
｝
//TMGTestScene.cpp:
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
// Identify and manipulate `eventType` here
}
Initializing the SDK
You need to initialize the SDK through the Init API before you can use the real-time voice, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
ITMGContext virtual int Init(const char* sdkAppId, const char* openId)
|
| | |
| | openID can only be in Int64 type, which is passed in after being converted to a const char*. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
Returned values
|
| Initialized SDK successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Checks whether the SDK file is complete. It is recommended to delete it and then import the SDK again. |
Notes on 7015 error code:
The 7015 error code is judged by md5. If this error is reported during integration, please check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Due to the third-party reinforcement, Unity packaging mechanism and other factors, the md5 of the library file will be affected, resulting in misjudgment. Please ignore this error in the logic for official release, and try to avoid displaying it in the UI.
Sample code
#define SDKAPPID3RD "14000xxxxx"
cosnt char* openId="10001";
ITMGContext* context = ITMGContextGetInstance();
context->Init(SDKAPPID3RD, openId);
Triggering event callback
Event callbacks can be triggered by periodically calling the Poll API in update. Poll is the message pump of GME, and the Poll API should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run exceptionally.\nYou can refer to the EnginePollHelper.cpp file in the demo.
Calling the `Poll` API periodically:
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
class ITMGContext {
protected:
virtual ~ITMGContext() {}
public:
virtual void Poll()= 0;
}
Sample code
// Declaration in the header file
class TMGTestScene : public cocos2d::Scene,public ITMGDelegate
{
void update(float delta);
}
// Code implementation
void TMGTestScene::update(float delta)
{
ITMGContextGetInstance()->Poll();
}
Pausing the system
When a Pause event occurs in the system, the engine should also be notified for pause. If you do not need the background to play back the audio in the room, please call Pause API to pause the GME service.
API prototype
Resuming the system
When a Resume event occurs in the system, the engine should also be notified for resumption. The Resume API only supports resuming voice chat.
API prototype
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->Uninit();
Voice Chat Room APIs
You should initialize and call the SDK to enter a room before voice chat can start.
|
| Calculates the local authentication key |
| |
| |
| Determines whether room entry is successful |
| Switches the room quickly |
Local authentication key calculation
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, please use the backend deployment key as detailed in Authentication Key. API prototype
int QAVSDK_AuthBuffer_GenAuthBuffer(unsigned int dwSdkAppID, const char* strRoomID, const char* strOpenID,
const char* strKey, unsigned char* strAuthBuffer, unsigned int bufferLength);
|
| | AppId from the Tencent Cloud console
|
| | Room ID, which can contain up to 127 characters. |
| | User ID, which is the same as openID during initialization. |
| | |
| | |
| | The length of the returned authbuff. 500 is recommended. |
Sample code
unsigned int bufferLen = 512;
unsigned char retAuthBuff[512] = {0};
QAVSDK_AuthBuffer_GenAuthBuffer(atoi(SDKAPPID3RD), roomId, "10001", AUTHKEY,retAuthBuff,bufferLen);
Entering a room
This API is used to enter a room with the generated authentication information. The mic and speaker are not enabled by default after room entry.
Note：
If the room entry callback result is 0, the room entry is successful. If 0 is returned from the EnterRoom API, it doesn't necessarily mean that the room entry is successful.
The audio type of the room is determined by the first user entering the room. After that, if a member in the room changes the room type, it will take effect for all members there. For example, if the first user entering the room uses the smooth sound quality, and the second user entering the room uses the HD sound quality, the room audio type of the second user will change to the smooth sound quality. Only after a member in the room calls the ChangeRoomType API will the audio type of the room be changed.
ITMGContext virtual int EnterRoom(const char* roomID, ITMG_ROOM_TYPE roomType, const char* authBuff, int buffLen)
|
| | Room ID, which can contain up to 127 characters. |
| | Room type. We recommend you select ITMG_ROOM_TYPE_FLUENCY for games. For more information on room audio types, see Sound Quality. |
| | |
| | Authentication key length |
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->EnterRoom(roomID, ITMG_ROOM_TYPE_FLUENCY, (char*)retAuthBuff,bufferLen);
Callback for room entry
After the user enters the room, the message ITMG_MAIN_EVENT_TYPE_ENTER_ROOM will be sent and identified in the OnEvent function for callback and processing. A successful callback means that the room entry is successful, and the billing starts.
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
}
}
Data details
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | | {"error_info":"waiting timeout, please check your network","result":0} |
If the network is disconnected, there will be a disconnected callback prompt ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT. At this time, the SDK will automatically reconnect, and the callback is ITMG_MAIN_EVENT_TYPE_RECONNECT_START. When the reconnection is successful, there will be a callback ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS.
Error codes
|
| Authentication failed: The AppID does not exist or is incorrect. An error occurred while authenticating the authbuff. Authentication expired. The OpenId does not meet the specification. |
| |
| The user was already in the process of entering a room but repeated this operation. It is recommended not to call the room entering API until the room entry callback is returned. |
| The user was already in the room and called the room entering API again. |
| Make sure that the SDK is initialized, OpenId complies with the rules, the APIs are called in the same thread, and the Poll API is called normally |
Exiting a room
This API is used to exit the current room. It is an async API. The returned value AV_OK indicates a successful async delivery. If there is a scenario in the application where room entry is performed immediately after room exit, you don't need to wait for the RoomExitComplete callback notification from the ExitRoom API during API call; instead, you can directly call the EnterRoom API.
API prototype
ITMGContext virtual int ExitRoom()
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->ExitRoom();
Callback for room exit
After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM.
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_EXIT_ROOM:
{
// Process
break;
}
}
}
Data details
|
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | | {"error_info":"","result":0} |
Determining whether a user has entered a room
This API is used to determine whether the user has entered a room. A value in bool type will be returned. Do not call this API during room entry.
API prototype
ITMGContext virtual bool IsRoomEntered()
Sample code
ITMGContext* context = ITMGContextGetInstance();
context->IsRoomEntered();
Switching room
User can call this API to quickly switch the voice chat room after entering the room. After the room is switched, the device is not reset, that is, if the microphone is already enabled in this room, the microphone will keep enabled after the room is switched.\nThe callback for quickly switching rooms is ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM, and the fields are error_info and result.
API prototype
ITMGContext virtual int SwitchRoom(const char* targetRoomID, const char* authBuff, int buffLen);
Type descriptions
|
| | |
| | Generates a new authentication key with the ID of the room to enter |
| | Authentication key length |
Room Status Maintenance
APIs in this section are used to display speaking members and members entering or exiting the room and mute a member in the room at the business layer.
|
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | The member status changed |
| Mutes a member in the room |
| |
Notification events of member room entry and speaking status
This API is used to obtain the user speaking in the room and display it in the UI, and to send a notification when someone entering or exiting the room.\nNotification for this event will be sent only when the status changes. To get member status in real time, cache the notification when it is received at a higher layer. The event message is ITMG_MAIN_EVNET_TYPE_USER_UPDATE, where the data contains event_id and user_list. The event message will be identified in the OnEvent function.\nNotifications for audio events are subject to a threshold, and a notification will be sent only when this threshold is exceeded. The notification "A member has stopped sending audio packets" will be sent if no audio packets are received in more than two seconds.
|
| Return the openid of the member entering the room. | |
| Return the openid of the member exiting the room. | |
ITMG_EVENT_ID_USER_HAS_AUDIO | Return the openid of the member sending audio packets in the room. This event can be used to determine whether a user is speaking and display the voiceprint effect. | |
ITMG_EVENT_ID_USER_NO_AUDIO | Return the openid of the member stopping sending audio packets in the room. | |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_USER_UPDATE:
{
// Process
// Parse the parameter to get `eventID` and `user_list`
switch (eventID)
{
case ITMG_EVENT_ID_USER_ENTER:
// A member enters the room
break;
case ITMG_EVENT_ID_USER_EXIT:
// A member exits the room
break;
case ITMG_EVENT_ID_USER_HAS_AUDIO:
// A member sends audio packets
break;
case ITMG_EVENT_ID_USER_NO_AUDIO:
// A member stops sending audio packets
break;
default:
break;
}
break;
}
}
}
Muting a member in the room
This API is used to add an ID to the audio data blocklist. This operation blocks audio from someone and only applies to the local device without affecting other devices. The returned value 0 indicates that the call is successful. Assume that users A, B, and C are all speaking using their mic in a room:
If A blocks C, A can only hear B;
If B blocks neither A nor C, B can hear both of them;
If C blocks neither A nor B, C can hear both of them.
This API is suitable for scenarios where a user is muted in a room.
API prototype
ITMGContext ITMGAudioCtrl int AddAudioBlackList(const char* openId)
|
| | openid of the user to be blocked
|
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->AddAudioBlackList(openId);
Unmuting
This API is used to remove an ID from the audio data blacklist. A returned value of 0 indicates the call is successful.
API prototype
ITMGContext ITMGAudioCtrl int RemoveAudioBlackList(const char* openId)
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->RemoveAudioBlackList(openId);
Voice Chat Capturing APIs
The voice chat APIs can only be called after SDK initialization and room entry.
When the user clicks the button of enabling/disabling the mic or speaker on the UI, we recommend you call the EnableMic or EnableSpeaker API.
To enable the user to press the mic button on the UI to speak and release it to stop speaking, we recommend you call EnableAudioCaptureDevice once during room entry and call EnableAudioSend to enable the user to speak while pressing the button.
|
| |
| |
| Enables/Disables the capturing device |
IsAudioCaptureDeviceEnabled | Gets the capturing device status |
| Enables/Disables audio upstreaming |
| Gets the audio upstreaming status |
| Gets the real-time mic volume level |
| Gets real-time audio upstreaming volume |
| Sets the mic volume level |
| Gets the mic volume level |
Enabling or disabling mic
This API is used to enable/disable the mic. The mic and speaker are not enabled by default after room entry. EnableMic = EnableAudioCaptureDevice + EnableAudioSend
API prototype
ITMGAudioCtrl virtual int EnableMic(bool bEnabled)
|
| | To enable the mic, set this parameter to true, otherwise, set it to false. |
Sample code
// Enable mic
ITMGContextGetInstance()->GetAudioCtrl()->EnableMic(true);
Getting the mic status
This API is used to get the mic status. The returned value 0 indicates that the mic is off, while 1 is on.
API prototype
ITMGAudioCtrl virtual int GetMicState()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicState();
Enabling or disabling capturing device
This API is used to enable/disable a capturing device. The device is not enabled by default after room entry.
This API can only be called after room entry. The device will be disabled automatically after room exit.
Operations such as permission application and volume type adjustment will generally be performed when a capturing device is enabled on a mobile device.
API prototype
ITMGAudioCtrl virtual int EnableAudioCaptureDevice(bool enable)
|
| | To enable the capturing device, set this parameter to true, otherwise, set it to false. |
Sample code
// Enable capturing device
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioCaptureDevice(true);
Getting the capturing device status
This API is used to get the status of a capturing device.
API prototype
ITMGContext virtual bool IsAudioCaptureDeviceEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioCaptureDeviceEnabled();
Enabling or disabling audio upstreaming
This API is used to enable/disable audio upstreaming. If a capturing device is already enabled, it will send captured audio data; otherwise, it will remain mute. For more information on how to enable/disable the capturing device, please see the EnableAudioCaptureDevice API.
API prototype
ITMGContext virtual int EnableAudioSend(bool bEnable)
|
| | To enable audio upstreaming, set this parameter to true; otherwise, set it to false. |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioSend(true);
Getting audio upstreaming status
This API is used to get the status of audio upstreaming.
API prototype
ITMGContext virtual bool IsAudioSendEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioSendEnabled();
Getting the real-time mic volume
This API is used to get the real-time mic volume. An int-type value in the range of 0-100 will be returned. It is recommended to call this API once every 20 ms.
API prototype
ITMGAudioCtrl virtual int GetMicLevel()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicLevel();
Getting the real-time audio upstreaming volume
This API is used to get the local real-time audio upstreaming volume. An int-type value in the range of 0-100 will be returned.
API prototype
ITMGAudioCtrl virtual int GetSendStreamLevel()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSendStreamLevel();
Setting the mic software volume
This API is used to set the mic volume level. The corresponding parameter is volume, which is equivalent to attenuating or gaining the captured sound.
API prototype
ITMGAudioCtrl virtual int SetMicVolume(int vol)
|
| | Value range: 0–200. Default value: 100. 0 indicates that the audio is mute, while 100 indicates that the volume level remains unchanged. |
Sample code
int micVol = (int)(value * 100);
ITMGContextGetInstance()->GetAudioCtrl()->SetMicVolume(vol);
Getting the mic software volume
This API is used to obtain the microphone volume. An "int" value is returned. Value 101 represents API SetMicVolume has not been called.
API prototype
ITMGAudioCtrl virtual int GetMicVolume()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicVolume();
Voice Chat Playback APIs
|
| Enables/Disables the speaker |
| |
| Enables/Disables the playback device |
| Gets playback device status |
| Enables/Disables audio downstreaming |
| Gets the audio downstreaming status |
| Gets the real-time speaker volume level |
| Gets the real-time downstreaming audio volume levels of other members in the room |
| Sets the speaker volume level |
| Gets the speaker volume level |
Enabling or disabling speaker
This API is used to enable/disable the speaker. EnableSpeaker = EnableAudioPlayDevice + EnableAudioRecv
API prototype
ITMGAudioCtrl virtual int EnableSpeaker(bool enable)
|
| | To disable the speaker, set this parameter to false; otherwise, set it to true. |
Sample code
// Enable the speaker
ITMGContextGetInstance()->GetAudioCtrl()->EnableSpeaker(true);
Getting the speaker status
This API is used to get the speaker status. 0 indicates that the speaker is off, and 1 is on.
API prototype
ITMGAudioCtrl virtual int GetSpeakerState()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerState();
Enabling or disabling playback device
This API is used to enable/disable a playback device.
API prototype
ITMGAudioCtrl virtual int EnableAudioPlayDevice(bool enable)
|
| | To disable the playback device, set this parameter to false; otherwise, set it to true. |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioPlayDevice(true);
Getting the playback device status
This API is used to get the status of a playback device.
API prototype
ITMGAudioCtrl virtual bool IsAudioPlayDeviceEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioPlayDeviceEnabled();
Enabling or disabling audio downstreaming
This API is used to enable/disable audio downstreaming. If a playback device is already enabled, it will play back audio data from other members in the room; otherwise, it will remain mute. For more information on how to enable/disable the playback device, please see the EnableAudioPlayDevice API.
API prototype
ITMGAudioCtrl virtual int EnableAudioRecv(bool enable)
|
| | To enable audio downstreaming, set this parameter to true; otherwise, set it to false. |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableAudioRecv(true);
Getting audio downstreaming status
This API is used to get the status of audio downstreaming.
API prototype
ITMGAudioCtrl virtual bool IsAudioRecvEnabled()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->IsAudioRecvEnabled();
Getting the real-time speaker volume
This API is used to get the real-time speaker volume. An int-type value will be returned to indicate the volume. It is recommended to call this API once every 20 ms.
API prototype
ITMGAudioCtrl virtual int GetSpeakerLevel()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerLevel();
Getting the real-time downstreaming audio levels of other members in the room
This API is used to get the real-time audio downstreaming volume of other members in the room. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGAudioCtrl virtual int GetRecvStreamLevel(const char* openId)
|
| | openId of other members in the room
|
Sample code
iter->second.level = ITMGContextGetInstance()->GetAudioCtrl()->GetRecvStreamLevel(iter->second.openid.c_str());
Setting the speaker volume
This API is used to set the speaker volume.
API prototype
ITMGAudioCtrl virtual int SetSpeakerVolume(int vol)
|
| | Volume level. Value range: 0–200. Default value: 100. 0 indicates that the audio is mute, while 100 indicates that the volume level remains unchanged. |
Sample code
int vol = 100;
ITMGContextGetInstance()->GetAudioCtrl()->SetSpeakerVolume(vol);
Getting the speaker volume
This API is used to get the speaker volume. An int-type value will be returned to indicate the volume. 101 indicates that the SetSpeakerVolume API has not been called.\n"Level" indicates the real-time volume, and "Volume" the speaker volume. The final volume = Level * Volume%. For example, if the "Level" is 100 and "Volume" is 60, the final volume is "60".
API prototype
ITMGAudioCtrl virtual int GetSpeakerVolume()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerVolume();
Device Selection APIs
Device selection APIs can be used only on PC.
|
| |
| |
| Gets the number of speakers |
| |
| |
| |
Getting the number of mics
This API is used to get the number of mics.
Function prototype
ITMGAudioCtrl virtual int GetMicListCount()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicListCount();
Enumerating mics
This API is used together with the GetMicListCount API to enumerate mics.
Function prototype
ITMGAudioCtrl virtual int GetMicList(TMGAudioDeviceInfo* ppDeviceInfoList, int nCount)
class TMGAudioDeviceInfo
{
public:
const char* pDeviceID;
const char* pDeviceName;
};
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetMicList(ppDeviceInfoList,nCount);
Selecting a mic
This API is used to select a mic. If this API is not called or DEVICEID_DEFAULT is passed in, the default mic will be selected.\nThe 0th device id returned in the GetMicList API is the default device of the call device. If there is a selected call device, it will be maintained by service. If it is unplugged, the call device will be changed back into the default device.
Function prototype
ITMGAudioCtrl virtual int SelectMic(const char* pMicID)
|
| | Mic ID, which is from the list returned by GetMicList. |
Sample code
const char* pMicID ="{0.0.1.00000000}.{7b0b712d-3b46-4f7a-bb83-bf9be4047f0d}";
ITMGContextGetInstance()->GetAudioCtrl()->SelectMic(pMicID);
Getting the number of speakers
This API is used to get the number of speakers.
Function prototype
ITMGAudioCtrl virtual int GetSpeakerListCount()
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerListCount();
Enumerating speakers
This API is used together with the GetSpeakerListCount API to enumerate speakers.
Function prototype
ITMGAudioCtrl virtual int GetSpeakerList(TMGAudioDeviceInfo* ppDeviceInfoList, int nCount)
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->GetSpeakerList(ppDeviceInfoList,nCount);
Selecting a speaker
This API is used to select a playback device. If this API is not called or DEVICEID_DEFAULT is passed in, the default playback device will be selected.
Function prototype
ITMGAudioCtrl virtual int SelectSpeaker(const char* pSpeakerID)
|
| | Speaker ID, which is from the list returned by GetSpeakerList. |
Sample code
const char* pSpeakerID ="{0.0.1.00000000}.{7b0b712d-3b46-4f7a-bb83-bf9be4047f0d}";
ITMGContextGetInstance()->GetAudioCtrl()->SelectSpeaker(pSpeakerID);
Advanced APIs
Enabling in-ear monitoring
This API is used to enable in-ear monitoring. You need to call EnableLoopBack+EnableSpeaker before you can hear your own voice.
API prototype
ITMGAudioCtrl virtual int EnableLoopBack(bool enable)
|
| | Specifies whether to enable |
Sample code
ITMGContextGetInstance()->GetAudioCtrl()->EnableLoopBack(true);
Getting a user's room audio type
This API is used to get a user's room audio type. The returned value is the room audio type. Value 0 indicates that an error occurred while getting the user's room audio type. For room audio types, please see the EnterRoom API.
API prototype
class ITMGRoom {
public:
virtual ~ITMGRoom() {} ;
virtual int GetRoomType() = 0;
};
Sample code
ITMGContext* context = ITMGContextGetInstance();
ITMGContextGetInstance()->GetRoom()->GetRoomType();
Changing the room type
This API is used to modify a user's room audio type. For the result, please see the callback event. The event type is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE. The audio type of the room is determined by the first user to enter the room. After that, if a member in the room changes the room type, it will take effect for all members there.
API prototype
IITMGContext TMGRoom public int ChangeRoomType((ITMG_ROOM_TYPE roomType)
|
| | Room type to be switched to the target type. For room audio types, please see the EnterRoom API. |
Sample code
ITMGContext* context = ITMGContextGetInstance();
ITMGContextGetInstance()->GetRoom()->ChangeRoomType(ITMG_ROOM_TYPE_FLUENCY);
Data details
|
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | result; error_info; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
Callback for room type setting completion
After the room type is set, the event message ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE will be returned in the callback. The returned parameters include result, error_info, and new_room_type. The new_room_type represents the following information. The event message will be identified in the OnEvent function.
|
ITMG_ROOM_CHANGE_EVENT_ENTERROOM | | Indicates that the existing audio type is inconsistent with and changed to that of the entered room. |
ITMG_ROOM_CHANGE_EVENT_START | | Indicates that a user is already in the room and the audio type starts changing (e.g., calling the ChangeRoomType API to change the audio type). |
ITMG_ROOM_CHANGE_EVENT_COMPLETE | | Indicates that a user is already in the room and the audio type has been changed. |
ITMG_ROOM_CHANGE_EVENT_REQUEST | | Indicates that a room member calls the ChangeRoomType API to request a change of room audio type. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
if (ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE == type)
{
// Process room type events
}
}
The monitoring event of room call quality
This is the quality monitoring event, which is triggered once every two seconds after room entry, and its message is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY. The returned parameters include weight, loss, and delay, which are as detailed below:
|
| | Value range: 1-50. 50 indicates excellent sound quality, 1 indicates very poor (barely usable) sound quality, and 0 represents an initial meaningless value. Generally, if the value is below 30, the business layer will remind users that the network is poor and recommend them to switch the network. |
| | Upstream packet loss rate |
| | |
Getting the version number
This API is used to get the SDK version number for analysis.
API prototype
ITMGContext virtual const char* GetSDKVersion()
Sample code
ITMGContextGetInstance()->GetSDKVersion();
Setting the log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
ITMGContext int SetLogLevel(ITMG_LOG_LEVEL levelWrite, ITMG_LOG_LEVEL levelPrint)
Parameter description
|
| | Sets the level of logs to be written. TMG_LOG_LEVEL_NONE indicates not to write. Default value: TMG_LOG_LEVEL_INFO |
| | Sets the level of logs to be printed. TMG_LOG_LEVEL_NONE indicates not to print. Default value: TMG_LOG_LEVEL_ERROR |
ITMG_LOG_LEVEL is as detailed below:
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
ITMGContextGetInstance()->SetLogLevel(TMG_LOG_LEVEL_INFO,TMG_LOG_LEVEL_INFO);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
|
| %appdata%\\Tencent\\GME\\ProcessName |
| Application/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/Documents |
| /sdcard/Android/data/xxx.xxx.xxx/files |
| /Users/username/Library/Containers/xxx.xxx.xxx/Data/Documents |
API prototype
ITMGContext virtual int SetLogPath(const char* logDir)
Sample code
cosnt char* logDir = ""// Set a path by yourself
ITMGContext* context = ITMGContextGetInstance();
context->SetLogPath(logDir);
Getting the diagnostic messages
This API is used to get information on the quality of real-time audio/video calls, which is mainly used to view real-time call quality and troubleshoot and can be ignored on the business side.
API prototype
ITMGRoom virtual const char* GetQualityTips()
Sample code
ITMGContextGetInstance()->GetRoom()->GetQualityTips();
Callback Messages
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | A member entered the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | A member exited the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | The room was disconnected for network or other reasons | result; error_info | {"error_info":"waiting timeout, please check your network","result":0} |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | The room members were updated | user_list; event_id | {"event_id":1,"user_list":["0"]} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_START | Room reconnection started | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS | Room reconnection succeeded | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM | The room was quickly switched | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | The room status changed | result; error_info; sub_event_type; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_START | Cross-room mic connect started | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_STOP | Cross-room mic connect stopped | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_DEFAULT_DEVICE_CHANGED | The default speaker was changed | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_NEW_DEVICE | A speaker was added | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_LOST_DEVICE | A speaker was lost | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_NEW_DEVICE | A mic was added | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"mic (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_LOST_DEVICE | A mic was lost | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_DEFAULT_DEVICE_CHANGED | The default mic was changed | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | Room quality message | weight; loss; delay | {"weight":5,"loss":0.1,"delay":1} |
ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE | Recording of a voice message was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE | Upload of a voice message was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE | Download of a voice message was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE | Playback of a voice message was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE | Fast recording-to-text conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE | Streaming speech-to-text conversion was completed | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING | A voice message is being converted into text in a streaming manner | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TEXT2SPEECH_COMPLETE | Text-to-speech conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TRANSLATE_TEXT_COMPLETE | Text translation was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
This document describes how to integrate with and debug GME client APIs for the voice messaging and speech-to-text services for Cocos2d.
Key Considerations for Using GME
GME provides the real-time voice service and voice messaging and speech-to-text services, which all depend on core APIs such as Init and Poll.
Caution:
There is a default call rate limit for speech-to-text APIs. For more information on how calls are billed within the limit, see Purchase Guide. If you want to increase the limit or learn more about how excessive calls are billed, submit a ticket. Non-streaming speech-to-text API SpeechToText(): There can be up to 10 concurrent requests per account.
Streaming speech-to-text API StartRecordingWithStreamingRecognition(): There can be up to 50 concurrent requests per account.
Real-time streaming speech-to-text API StartRealTimeASR(): There can be up to 50 concurrent requests per account.
Notes
You have created a GME application and obtained the SDK AppID and key. For more information, see Activating Services. You have activated GME real-time voice service and voice messaging and speech-to-text services. For more information, see Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, AV_OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Integrating the SDK
Directions
Key processes involved in SDK integration are as follows:
C++ classes
|
| |
| Voice messaging and speech-to-text APIs |
Key APIs
#include "auth_buffer.h"
#include "tmg_sdk.h"
#include "AdvanceHeaders/tmg_sdk_adv.h"
#include <vector>
Callback
Setting callback sample
// When initializing the SDK
m_pTmgContext = ITMGContextGetInstance();
m_pTmgContext->SetTMGDelegate(this);
// In the destructor
CTMGSDK_For_AudioDlg::~CTMGSDK_For_AudioDlg()
{
ITMGContextGetInstance()->SetTMGDelegate(NULL);
}
Message delivery
The API class uses the Delegate method to send callback notifications to the application. ITMG_MAIN_EVENT_TYPE indicates the message type. The data on Windows is in json string format. For the key-value pairs, please see the relevant documentation.
// Declaration in the header file
virtual void OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data);
// Sample code
void CTMGSDK_For_AudioDlg::OnEvent(ITMG_MAIN_EVENT_TYPE eventType, const char* data)
{
switch(eventType)
{
case ITMG_MAIN_EVENT_TYPE_XXXX_XXXX:
{
// Process the callback
}
break;
}
}
Getting singleton
The GME SDK is provided in the form of a singleton, all calls begin with ITMGContext, and callbacks are passed to the application through ITMGDelegate, which should be configured first.
Sample code
ITMGContext* m_pTmgContext;
m_pTmgContext->Init(AppID, OpenID);
You need to initialize the SDK through the Init API before you can use the real-time voice, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
ITMGContext virtual int Init(const char* sdkAppId, const char* openId)
|
| | |
| | openID can only be in Int64 type, which is passed in after being converted to a const char*. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
Returned values
|
| Initialized SDK successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Checks whether the SDK file is complete. It is recommended to delete it and then import the SDK again. |
Notes on 7015 error code:
The 7015 error code is judged by md5. If this error is reported during integration, please check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Due to the third-party reinforcement, Unity packaging mechanism and other factors, the md5 of the library file will be affected, resulting in misjudgment. Please ignore this error in the logic for official release, and try to avoid displaying it in the UI.
Sample code
#define SDKAPPID3RD "14000xxxxxx"
cosnt char* openId="10001";
ITMGContext* context = ITMGContextGetInstance();
context->Init(SDKAPPID3RD, openId);
Triggering event callback
Event callbacks can be triggered by periodically calling the Poll API in update. Poll is the message pump of GME, and the Poll API should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run exceptionally.\nYou can refer to the EnginePollHelper.cpp file in the demo.
Note:
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
class ITMGContext {
protected:
virtual ~ITMGContext() {}
public:
virtual void Poll()= 0;
}
Sample code
void TMGTestScene::update(float delta)
{
ITMGContextGetInstance()->Poll();
}
Pausing the system
When a Pause event occurs in the system, the engine should also be notified for pause. If you do not need the background to play back the audio in the room, please call Pause API to pause the GME service.
API prototype
Resuming the system
When a Resume event occurs in the system, the engine should also be notified for resumption. The Resume API only supports resuming voice chat.
API prototype
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Messaging and Speech-to-Text Services
Note:
The speech-to-text service consists of fast recording-to-text conversion and streaming speech-to-text conversion.
You do not need to enter a voice chat room when using the voice message service.
The maximum recording duration of a voice message is 58 seconds by default, and the minimum recording duration cannot be less than 1 second. If you want to customize the recording duration, for example, to modify the maximum recording duration to 10 seconds, call the SetMaxMessageLength API to set it after initialization.
Flowchart for using the speech-to-text service
|
| Generates the local authentication key |
| Initializes authentication |
| Specifies the maximum length of voice message |
Generating the local authentication key
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, please use the backend deployment key as detailed in Authentication Key. API prototype
int QAVSDK_AuthBuffer_GenAuthBuffer(unsigned int dwSdkAppID, const char* strRoomID, const char* strOpenID,
const char* strKey, unsigned char* strAuthBuffer, unsigned int bufferLength);
|
| | AppId from the Tencent Cloud console.
|
| | Enter null or an empty string |
| | User Identifier, which is the same as openID during initialization. |
| | Permission key from the Tencent Cloud console. |
| | |
| | Length of the authbuff passed in. 500 is recommended. |
Application authentication
After the authentication information is generated, the authentication is assigned to the SDK.
API prototype
ITMGPTT virtual int ApplyPTTAuthbuffer(const char* authBuffer, int authBufferLen)
Sample code
ITMGContextGetInstance()->GetPTT()->ApplyPTTAuthbuffer(authBuffer,authBufferLen);
Specifying the maximum duration of voice message
This API is used to specify the maximum duration of a voice message, which can be up to 58 seconds.
API prototype
ITMGPTT virtual int SetMaxMessageLength(int msTime)
|
| | Audio duration in ms. Value range: 1000 < msTime <= 58000 |
Sample code
int msTime = 10000;
ITMGContextGetInstance()->GetPTT()->SetMaxMessageLength(msTime);
Streaming Speech Recognition
Voice messaging and speech-to-text APIs
|
StartRecordingWithStreamingRecognition | Starts streaming recording |
| Stops recording |
Starting streaming speech recognition
This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. It can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation. To stop recording, call Stop recording. API prototype
ITMGPTT virtual int StartRecordingWithStreamingRecognition(const char* filePath)
ITMGPTT virtual int StartRecordingWithStreamingRecognition(const char* filePath,const char* translateLanguage,const char* translateLanguage)
|
| | Path of stored audio file |
| | |
| | |
Sample code
ITMGContextGetInstance()->GetPTT()->StartRecordingWithStreamingRecognition(filePath,"cmn-Hans-CN","cmn-Hans-CN");
Note:
Translation incurs additional fees. For more information, see Purchase Guide. Callback for streaming speech recognition
After streaming speech recognition is started, you need to listen on callback messages in the OnEvent notification, which is as detailed below:
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING returns the recognized text in real-time during the recording, which is equivalent to returning the recognized text while speaking.
The event message will be identified in the OnEvent notification based on the actual needs. The passed parameters include the following four messages.
|
| Return code indicating whether streaming speech recognition is successful |
| Text converted from speech |
| Local path of stored recording file |
| Backend URL address of recording file, which will be retained for 90 days |
Note:
The file_id is empty when the 'ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRecognition_IS_RUNNING' message is listened.
Error codes
|
| Streaming speech-to-text conversion failed, but recording succeeded. | Call the UploadRecordedFile API to upload the recording file and then call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed, but recording and upload succeeded. | The message returned contains a backend URL after successful upload. Call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed. | During streaming recording, wait for the execution result of the streaming recording API to return. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
If the error code 4098 is reported, see FAQs for solutions. Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE:
{
HandleSTREAM2TEXTComplete(data,true);
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING:
{
HandleSTREAM2TEXTComplete(data, false);
break;
}
}
}
void CTMGSDK_For_AudioDlg::HandleSTREAM2TEXTComplete(const char* data, bool isComplete)
{
std::string strText = "STREAM2TEXT: ret=";
strText += data;
m_EditMonitor.SetWindowText(MByteToWChar(strText).c_str());
Json::Reader reader;
Json::Value root;
bool parseRet = reader.parse(data, root);
if (!parseRet) {
::SetWindowText(m_EditInfo.GetSafeHwnd(),MByteToWChar(std::string("parse result Json error")).c_str());
}
else
{
if (isComplete) {
::SetWindowText(m_EditUpload.GetSafeHwnd(), MByteToWChar(root["file_id"].asString()).c_str());
}
else {
std::string isruning = "STREAMINGRECOGNITION_IS_RUNNING";
::SetWindowText(m_EditUpload.GetSafeHwnd(), MByteToWChar(isruning).c_str());
}
}
}
Voice Message Recording
The recording process is as follows: start recording > stop recording > return recording callback > start the next recording.
Voice message and speech-to-text APIs
Starting recording
This API is used to start recording.
API prototype
ITMGPTT virtual int StartRecording(const char* fileDir)
|
| | Path of stored audio file |
Sample code
char buffer[256]={0};
snprintf(buffer, sizeof(buffer), "%sunreal_ptt_local.file", getFilePath().c_str());
ITMGContextGetInstance()->GetPTT()->StartRecording(buffer);
Stopping recording
This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.
API prototype
ITMGPTT virtual int StopRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->StopRecording();
Callback for recording start
The recording start result will be returned through the callback.
To stop recording, call StopRecording. The callback for recording start will be returned after the recording is stopped.
|
| | 0: recording is completed |
| | Path of stored recording file, which must be accessible and cannot be the fileid |
Error codes
|
| | Check whether the API parameters in the code are correct. |
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| Recording is in progress. | Ensure that the SDK recording feature is used at the right time. |
| Audio data is not captured. | Check whether the mic is working properly. |
| An error occurred while accessing the file during recording. | Ensure the existence of the file and the validity of the file path. |
| The mic is not authorized. | Mic permission is required for using the SDK. To add the permission, please see the SDK project configuration document for the corresponding engine or platform. |
| The recording duration is too short. | The recording duration should be in ms and longer than 1,000 ms. |
| No recording operation is started. | Check whether the recording starting API has been called. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE:
{
// Process
break;
}
}
}
Pausing recording
This API is used to pause recording. If you want to resume recording, please call the ResumeRecording API.
API prototype
ITMGPTT virtual int PauseRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->PauseRecording();
Resuming recording
This API is used to resume recording.
API prototype
ITMGPTT virtual int ResumeRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->ResumeRecording();
Canceling recording
This API is used to cancel recording. There is no callback after cancellation.
API prototype
ITMGPTT virtual int CancelRecording()
Sample code
ITMGContextGetInstance()->GetPTT()->CancelRecording();
Voice Message Upload, Download, and Playback
|
| |
| |
| Plays back the audio file |
| Stops playing back the audio file |
| |
| Gets the audio file duration |
Uploading an audio file
This API is used to upload an audio file.
API prototype
ITMGPTT virtual int UploadRecordedFile(const char* filePath)
|
| | Path of uploaded audio file, which is a local path |
Sample code
ITMGContextGetInstance()->GetPTT()->UploadRecordedFile(filePath);
Callback for audio file upload completion
After the audio file is uploaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
| | 0: recording is completed |
| | Path of stored recording file |
| | |
Error codes
|
| An error occurred while accessing the file during upload. | Ensure the existence of the file and the validity of the file path. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| A network error occurred. | Check whether the device can access the internet. |
| The network failed while getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| The packet returned during the process of getting the upload parameters is empty. | Check whether the authentication is correct and whether the device network can normally access the internet. |
| Failed to decode the packet returned during the process of getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| | Check whether the apply API is called or whether the input parameters are empty. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE:
{
// Process
break;
}
}
}
Downloading the audio file
This API is used to download an audio file.
API prototype
ITMGPTT virtual int DownloadRecordedFile(const char* fileId, const char* filePath)
Sample code
ITMGContextGetInstance()->GetPTT()->DownloadRecordedFile(fileID,filePath);
Callback for audio file download completion
After the audio file is downloaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
| | 0: recording is completed |
| | Path of stored recording file |
| | URL path of file, which will be retained on the server for 90 days |
Error codes
|
| An error occurred while accessing the file during download. | Check whether the file path is valid. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| Network storage system exception. | The server failed to get the audio file. Check whether the API parameter fileid is correct, whether the network is normal, and whether the file exists in COS. |
| Server file system error. | Check whether the device can access the internet and whether the file exists on the server. |
| The HTTP network failed during the process of getting the download parameters. | Check whether the device can access the internet. |
| The packet returned during the process of getting the download parameters is empty. | Check whether the device can access the internet. |
| Failed to decode the packet returned during the process of getting the download parameters. | Check whether the device can access the internet. |
| | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
Playing back audio
This API is used to play back audio.
API prototype
ITMGPTT virtual int PlayRecordedFile(const char* filePath)
ITMGPTT virtual int PlayRecordedFile(const char* filePath, nt voiceType)
Error codes
|
| | Ensure the existence of the file and the validity of the file path. |
Sample code
ITMGContextGetInstance()->GetPTT()->PlayRecordedFile(filePath);
Callback for audio playback
After the audio is played back, the event message ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameter includes result and file_path.
|
| | |
| | Path of stored recording file |
Error codes
|
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| During playback, the client tried to interrupt and play back the next one but failed (which should succeed normally). | Check whether the code logic is correct. |
| | Check whether the API parameters in the code are correct. |
| | An error occurred while initializing the player. This error code is generally caused by failure in decoding, and the error should be located with the aid of logs. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
...
else if (eventType == ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE) {
int32 result = JsonObject->GetIntegerField(TEXT("result"));
FString filepath = JsonObject->GetStringField(TEXT("file_path"));
onPttPlayFileCompleted(result, filepath);
}
}
}
Stopping audio playback
This API is used to stop audio playback. There will be a callback for playback completion when the playback stops.
API prototype
ITMGPTT virtual int StopPlayFile()
Sample code
ITMGContextGetInstance()->GetPTT()->StopPlayFile();
Getting audio file size
This API is used to get the size of an audio file.
API prototype
ITMGPTT virtual int GetFileSize(const char* filePath)
|
| | Path of audio file, which is a local path |
Sample code
ITMGContextGetInstance()->GetPTT()->GetFileSize(filePath);
Getting audio file duration
This API is used to get the duration of an audio file in milliseconds.
API prototype
ITMGPTT virtual int GetVoiceFileDuration(const char* filePath)
|
| | Path of audio file, which is a local path |
Sample code
ITMGContextGetInstance()->GetPTT()->GetVoiceFileDuration(filePath);
Fast Recording-to-Text Conversion
Converting audio file to text
This API is used to convert a specified audio file to text.
API prototype
ITMGPTT virtual void SpeechToText(const char* fileID)
Sample code
ITMGContextGetInstance()->GetPTT()->SpeechToText(fileID);
Translating audio file into text in specified language
This API can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation.
Note:
Translation incurs additional fees. For more information, see Purchase Guide. API prototype
ITMGPTT virtual int SpeechToText(const char* fileID,const char* speechLanguage)
ITMGPTT virtual int SpeechToText(const char* fileID,const char* speechLanguage,const char* translateLanguage)
|
| | The URL of the audio file, which will be retained on the server for 90 days. |
| | |
| | |
Sample code
ITMGContextGetInstance()->GetPTT()->SpeechToText(filePath,"cmn-Hans-CN","cmn-Hans-CN");
Callback for recognition
After the specified audio file is converted to text, the event message ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path and text (recognized text).
|
| | 0: recording is completed |
| | URL of recording file, which will be retained on the server for 90 days |
| | |
Error codes
|
| An internal error occurred. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the device can access the internet. |
| Failed to decode the returned packet. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
| | Check whether authbuffer is correct. |
| Incorrect speech-to-text conversion parameter. | Check whether the API parameter fileid in the code is empty. |
| Speech-to-text translation returned an error. | Error with the backend of voice messaging and speech-to-text feature. Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
void TMGTestScene::OnEvent(ITMG_MAIN_EVENT_TYPE eventType,const char* data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE:
{
// Process
break;
}
}
}
Voice Message Volume Level APIs
|
| Gets real-time mic volume level |
| Sets recording volume level |
| Gets recording volume level |
| Gets real-time speaker volume |
| Sets playback volume level |
| Gets playback volume level |
Getting the real-time mic volume of voice message
This API is used to get the real-time mic volume. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetMicLevel()
Sample code
ITMGContext.GetInstance(this).GetPTT().GetMicLevel();
Setting the recording volume of voice message
This API is used to set the recording volume of voice message. Value range: 0-200.
API prototype
ITMGPTT virtual int SetMicVolume(int vol)
Sample code
ITMGContextGetInstance()->GetPTT()->SetMicVolume(100);
Getting the recording volume of voice message
This API is used to get the recording volume of voice message. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetMicVolume()
Sample code
ITMGContextGetInstance()->GetPTT()->GetMicVolume();
Getting the real-time speaker volume of voice message
This API is used to get the real-time speaker volume. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetSpeakerLevel()
Sample code
ITMGContextGetInstance()->GetPTT()->GetSpeakerLevel();
Setting the playback volume of voice message
This API is used to set the playback volume of voice message. Value range: 0-200.
API prototype
ITMGPTT virtual int SetSpeakerVolume(int vol)
Sample code
ITMGContextGetInstance()->GetPTT()->SetSpeakerVolume(100);
Getting the playback volume of voice message
This API is used to get the playback volume of voice message. An int-type value will be returned. Value range: 0-200.
API prototype
ITMGPTT virtual int GetSpeakerVolume()
Sample code
ITMGContextGetInstance()->GetPTT()->GetSpeakerVolume();
Advanced APIs
Getting version number
This API is used to get the SDK version number for SDK usage analysis.
API prototype
ITMGContext virtual const char* GetSDKVersion()
Sample code
ITMGContextGetInstance()->GetSDKVersion();
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
ITMGContext int SetLogLevel(ITMG_LOG_LEVEL levelWrite, ITMG_LOG_LEVEL levelPrint)
Parameter description
|
| | Sets the level of logs to be written. TMG_LOG_LEVEL_NONE indicates not to write. Default value: TMG_LOG_LEVEL_INFO |
| | Sets the level of logs to be printed. TMG_LOG_LEVEL_NONE indicates not to print. Default value: TMG_LOG_LEVEL_ERROR |
ITMG_LOG_LEVEL description:
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
ITMGContextGetInstance()->SetLogLevel(TMG_LOG_LEVEL_INFO,TMG_LOG_LEVEL_INFO);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
|
| %appdata%\\Tencent\\GME\\ProcessName |
| Application/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/Documents |
| /sdcard/Android/data/xxx.xxx.xxx/files |
| /Users/username/Library/Containers/xxx.xxx.xxx/Data/Documents |
API prototype
ITMGContext virtual int SetLogPath(const char* logDir)
Sample code
cosnt char* logDir = ""// Set a path by yourself
ITMGContext* context = ITMGContextGetInstance();
context->SetLogPath(logDir);
Callback Messages
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | A member entered the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | A member exited the audio room | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | The room was disconnected for network or other reasons | result; error_info | {"error_info":"waiting timeout, please check your network","result":0} |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | Room members were updated | user_list; event_id | {"event_id":1,"user_list":["0"]} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_START | The reconnection to the room started | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS | The reconnection to the room succeeded | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM | The room was quickly switched | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | The room status was changed | result; error_info; sub_event_type; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_START | Cross-room mic connect started | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_STOP | Cross-room mic connect stopped | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_DEFAULT_DEVICE_CHANGED | The default speaker device was changed | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_NEW_DEVICE | A new speaker device was added | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_LOST_DEVICE | A speaker device was lost | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_NEW_DEVICE | A new mic device was added | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_LOST_DEVICE | A mic device was lost | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_DEFAULT_DEVICE_CHANGED | The default mic device was changed | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | The room quality changed | weight; loss; delay | {"weight":5,"loss":0.1,"delay":1} |
ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE | Voice message recording was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE | Voice message upload was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE | Voice message download was completed | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE | Voice message playback was completed | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE | Fast speech-to-text conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE | Streaming speech-to-text conversion was completed | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING | Streaming speech-to-text conversion is in progress | result; file_path; text;file_id | {"file_id":"","file_path":","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TEXT2SPEECH_COMPLETE | Text-to-speech conversion was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_TRANSLATE_TEXT_COMPLETE | Text translation was completed | result; text;file_id | {"file_id":"","text":"","result":0} |
This document describes how to configure a Flutter project for the GME APIs for Flutter.
The GME SDK for Flutter supports iOS and Android platforms.
Importing the SDK
Step 1. Download the GME SDK for Flutter
Download the SDK file in SDK Download Guide, which contains the GME plugin. Decompress the SDK file to a local directory. Step 2. Add dependencies of the GME plugin to the Flutter project
Add GME dependencies to the pubspec.yaml file in your Flutter project. Note that the path parameter is the path where the SDK file is decompressed to.
dependencies:
flutter:
sdk: flutter
gme:
path: ../
After saving the pubspec.yaml file, run the flutter pub get command on the CLI to make the GME plugin in the project take effect (if the Flutter plugin is configured in your IDE, this command will be executed automatically once the file is saved).
Modifying the iOS Project
1. In Terminal, go to the iOS project directory of your Flutter project and run pod install.
2. In the Xcode project, configure the following GME dependent library files (you can skip this step if such dependencies already exist in your project):
3. The GME SDK for iOS requires the following permissions:
Required background modes: Allows running in the background (optional).
Microphone Usage Description: Allows access to microphone.
Modifying the Android Project
1. As GME requires permissions such as call permission and uses the permission management plugin flutter permission-handler, you need to modify the project as follows to use the SDK for Android 31 or later (skip this step if the SDK is already in the project):
2. Add the project permissions to the Flutter project file android/app/src/AndroidManifest.xml (skip this step if such permissions have been added):
<uses-permission android:name="android.permission.RECORD_AUDIO" />
<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
<uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.MODIFY_AUDIO_SETTINGS" />
This document describes how to integrate with and debug GME client APIs for the real-time voice chat feature for Flutter.
Key Considerations for Using GME
GME provides the real-time voice chat service, voice messaging and speech-to-text services, which all depend on core APIs such as Init and Poll.
Notes
You have created a GME application and obtained the SDK AppID and key. For more information, see Activating Services. You have activated GME real-time voice chat, voice messaging and speech-to-text services. For more information, see Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, GmeError.AV_OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Integrating the SDK
Directions
Key processes involved in SDK integration are as follows:
Core APIs
|
Init | Initializes GME. |
Poll | Triggers the event callback. |
Pause | Pauses the system. |
Resume | Resumes the system. |
Uninit | Uninitializes GME. |
Importing the GME module
import 'package:gme/gme.dart';
import 'package:gme/gmeType.dart';
Getting an instance
To use the voice chat feature, get the GmeSDK object first.
ITMGContext context = ITMGContext.GetInstance();
Initializing the SDK
You need to initialize the SDK through the Init API before you can use the real-time voice, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
//class ITMGContext
Future<int> InitSDK(String appID, String openID)
|
sdkAppId | string | |
openID | string | `openID` can only be in `int64` type, which is passed in after being converted to a string. You can customize its rules, and it must be unique in the application. To pass in `openID` as a string, submit a ticket for application. |
Returned values
|
GmeError.AV_OK= 0 | SDK initialized successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Solution: Check whether the SDK file is complete. We recommend that you delete it and then import the SDK again. |
Note:
Notes on 7015 error code
The 7015 error code is identified by MD5. If this error is reported during integration, check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Due to the third-party enhancement, the Unity packaging mechanism, and other factors, the MD5 value of the library file will be affected, resulting in misjudgment. Therefore, ignore this error in the logic for official releases, and avoid displaying it on the UI.
Sample code
string SDKAPPID3RD = "14000xxxxx";
string openId="10001";
int res = await ITMGContext.GetInstance().InitSDK(SDKAPPID3RD, openId);
if (ret != GmeError.AV_OK)
{
print("Init SDK Error");
return;
}
Setting callbacks
The API class uses the Delegate method to send callback notifications to the application. Register the callback function to the SDK for receiving callback messages before room entry.
Function prototype and sample code
Register the callback function to the SDK for receiving callback messages before room entry.
// When initializing the SDK
ITMGContext.GetInstance().SetEvent(handleEventMsg);
// Callback method
void handleEventMsg(int eventType, String data) async {
// enterRoom event
print("AddDelegate3" + eventType.toString());
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Callback of room entry\t
}
break;
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE:
{
// Callback of room switch
}
break;
}
}
Triggering event callback
You need to periodically call the Poll API to trigger event callbacks. The Poll API is GME's message pump and should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run abnormally. For more information, see the EnginePollHelper file in SDK Download Guide. Note:
Call the Poll API periodically
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
Sample code
Future<void> pollTimer() async {_pollTimer = Timer.periodic(Duration(milliseconds: 100), (Timer timer) {
ITMGContext.GetInstance().Poll();
});
}
Pausing the system
When a Pause event occurs in the system, the engine should also be notified for pause. For example, when the application switches to the background (OnApplicationPause, isPause=True), and you do not need the background to play back the audio in the room, please call Pause API to pause the GME service.
API prototype
Resuming the system
When a Resume event occurs in the system, the engine should also be notified for resumption. The Resume API only supports resuming voice chat.
API prototype
Uninitializing SDK
This API is used to uninitialize the SDK. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Chat Room APIs
You should initialize and call the SDK to enter a room before voice chat can start.\nIf you have any questions when using the service, see Sound and Audio. |
GenAuthBuffer | Calculates the local authentication key. |
EnterRoom | Enters a room. |
ExitRoom | Exits a room. |
IsRoomEntered | Determines whether room entry is successful. |
Local authentication key calculation
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, use the backend deployment key as detailed in Authentication Key. API prototype
Future<Uint8List> GenAuthBuffer(String appID, String roomID, String openID, String key)
|
appID | string | `AppID` from the Tencent Cloud console |
roomID | string | Room ID, which can contain up to 127 characters. |
openID | string | User ID, which is the same as `openID` during initialization. |
key | string | Permission key from the Tencent Cloud console. |
Sample code
Uint8List userSig = await ITMGContext.GetInstance().GenAuthBuffer(_editAppID.text, _editRoomID.text, _editOpenID.text, _editKey.text);
int res = await ITMGContext.GetInstance().EnterRoom(_editRoomID.text, 1, userSig);
Entering a room
This API is used to enter a room with the generated authentication information. The mic and speaker are not enabled by default after room entry.
Note:
If the room entry callback result is 0, the room entry is successful. If 0 is returned from the EnterRoom API, it doesn't necessarily mean that the room entry is successful.
The audio type of the room is determined by the first user entering the room. After that, if a member in the room changes the room type, it will take effect for all members there. For example, if the first user entering the room uses the smooth sound quality, and the second user entering the room uses the HD sound quality, the room audio type of the second user will change to the smooth sound quality. Only after a member in the room calls the ChangeRoomType API will the audio type of the room be changed.
API prototype
Future<int> EnterRoom(String roomID, int roomType, Uint8List authBuffer)
|
roomId | string | Room ID, which can contain up to 127 characters. |
roomType | ITMGRoomType | Room type. We recommend that you select `ITMG_ROOM_TYPE_FLUENCY` for games. For more information on room audio types, see Sound Quality. |
appKey | Uint8List | Authentication key |
Sample code
int res = await ITMGContext.GetInstance().EnterRoom(_editRoomID.text, 1, authBuffer);
Callback for room entry
After the user enters the room, the ITMG_MAIN_EVENT_TYPE_ENTER_ROOM event type will be called back to notify the room entry result, which can be listened on for processing. A successful callback means that the room entry is successful, and the billing starts.
Note:
Billing references:
Sample code
// Listen on an event:
void handleEventMsg(int eventType, String data) async {
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// The process after room entry
}
}
}
ITMGContext.GetInstance().SetEvent(handleEventMsg);
Data details
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | result; error_info | {"error_info":"waiting timeout, please check your network","result":0} |
If the network is disconnected, there will be a disconnection callback notification ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT. At this time, the SDK will automatically reconnect, and the callback is ITMG_MAIN_EVENT_TYPE_RECONNECT_START. When the reconnection is successful, there will be a callback ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS.
Error codes
|
7006 | Authentication failed. Possible causes: - The `AppID` does not exist or is incorrect.- An error occurred while authenticating the `authbuff`.- Authentication expired.- The `OpenId` does not meet the specification. |
7007 | The user was already in another room. |
1001 | The user was already in the process of entering a room but repeated this operation. We recommend that you not call the room entering API until the room entry callback is returned. |
1003 | The user was already in the room and called the room entering API again. |
1101 | Make sure that the SDK is initialized, `OpenId` complies with the rules, the APIs are called in the same thread, and the `Poll` API is called normally. |
Exiting a room
This API is used to exit the current room. It is an async API. The returned value AV_OK indicates a successful async delivery. If there is a scenario in the application where room entry is performed immediately after room exit, you don't need to wait for the RoomExitComplete callback notification from the ExitRoom API; instead, you can directly call the EnterRoom API.
API prototype
Sample code
ITMGContext.GetInstance().ExitRoom();
Callback for room exit
After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM. The sample code is shown below:
Sample code
void handleEventMsg(int eventType, String data) async{
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_EXIT_ROOM:
{
// The process after room exit
break;
}
}
}
ITMGContext.GetInstance().SetEvent(handleEventMsg);
Determining whether user has entered room
This API is used to determine whether the user has entered a room. A value in bool type will be returned. Do not call this API during room entry.
API prototype
Future<bool> IsRoomEntered()
Sample code
bool res = await ITMGContext.GetInstance().IsRoomEntered();
Room Status Maintenance
APIs in this section are used to display speaking members and members entering or exiting the room and mute a member in the room at the business layer.
|
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | The member status changed. |
AddAudioBlackList | Mutes a member in the room. |
RemoveAudioBlackList | Unmutes a user. |
IsOpenIdInAudioBlackList | Queries whether the user of the specified `openid` is muted. |
Notification events of member room entry and speaking status
This event is used to get speaking users in the room and display the users on the UI, and to send a notification when someone enters or exits the room.
A notification for this event will be sent only when the status changes. To get the member status in real time, cache the notification when it is received at the business layer. The event message ITMG_MAIN_EVNET_TYPE_USER_UPDATE containing event_id, count, and openIdList will be returned, which will be identified in the OnEvent notification.
Notifications of the EVENT_ID_ENDPOINT_NO_AUDIO audio event will be sent only when the threshold is exceeded, that is, other members in the room can receive the notification that the local user stops speaking only after the local client captures no voice for two seconds.
The audio event returns only the member speaking status but not the specific volume level. If you need the specific volume levels of members in the room, you can use the GetVolumeById API.
|
EVENT_ID_ENDPOINT_ENTER | Return the `openid` of the member entering the room. | Member list |
EVENT_ID_ENDPOINT_EXIT | Return the `openid` of the member exiting the room. | Member list |
EVENT_ID_ENDPOINT_HAS_AUDIO | Return the `openid` of the member sending audio packets in the room. This event can be used to determine whether a user is speaking and display the voiceprint effect. | Chat member list |
EVENT_ID_ENDPOINT_NO_AUDIO | Return the `openid` of the member stopping sending audio packets in the room. | Chat member list |
Sample code
void handleEventMsg(int eventType, String data) async {
if (eventType == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
{
// Process
switch (eventID)
{
case EVENT_ID_ENDPOINT_ENTER:
// A member enters the room
break;
case EVENT_ID_ENDPOINT_EXIT:
// A member exits the room
break;
case EVENT_ID_ENDPOINT_HAS_AUDIO:
// A member sends audio packets
break;
case EVENT_ID_ENDPOINT_NO_AUDIO:
// A member stops sending audio packets
break;
default:
break;
}
break;
}
}
Muting a member in the room
This API is used to add an ID to the audio data blocklist. This operation blocks audio from someone and only applies to the local device without affecting other devices. The returned value 0 indicates that the call is successful. Assume that users A, B, and C are all speaking using their mic in a room:
If A blocks C, A can only hear B;
If B blocks neither A nor C, B can hear both of them;
If C blocks neither A nor B, C can hear both of them.
This API is suitable for scenarios where a user is muted in a room.
API prototype
Future<int> AddAudioBlackList(String openID)
|
openID | string | `openid` of the user to be blocked |
Sample code
res = await ITMGContext.GetInstance().GetAudioCtrl().AddAudioBlackList(_editRoomManagerID.text);
Unmuting
This API is used to remove an ID from the audio data blocklist. A returned value of 0 indicates the call is successful.
API prototype
Future<int> RemoveAudioBlackList(String openID)
|
openId | string | ID to be unblocked |
Sample code
res = await ITMGContext.GetInstance().GetAudioCtrl().RemoveAudioBlackList(_editRoomManagerID.text);
Voice Chat Capturing APIs
The voice chat APIs can only be called after SDK initialization and room entry.
When the user clicks the button of enabling/disabling the mic or speaker on the UI, we recommend you call the EnableMic or EnableSpeaker API.
To enable the user to press the mic button on the UI to speak and release it to stop speaking, we recommend you call EnableAudioCaptureDevice once during room entry and call EnableAudioSend to enable the user to speak while pressing the button.
|
EnableMic | Enables/Disables the mic. |
GetMicState | Gets the mic status. |
EnableAudioCaptureDevice | Enables/Disables the capturing device. |
IsAudioCaptureDeviceEnabled | Gets the capturing device status. |
EnableAudioSend | Enables/Disables audio upstreaming. |
IsAudioSendEnabled | Gets the audio upstreaming status. |
GetMicLevel | Gets the real-time mic volume level. |
GetSendStreamLevel | Gets the real-time audio upstreaming volume level. |
SetMicVolume | Sets the mic volume level. |
GetMicVolume | Gets the mic volume level. |
Enabling or disabling mic
This API is used to enable/disable the mic. The mic and speaker are not enabled by default after room entry. EnableMic = EnableAudioCaptureDevice + EnableAudioSend
API prototype
Future<int> EnableMic(bool enable)
|
isEnabled | bool | To enable the mic, set this parameter to true; otherwise, set it to false. |
Sample code
// Turn on mic
int res = await ITMGContext.GetInstance().GetAudioCtrl().EnableMic(true);
Getting the mic status
This API is used to get the mic status. The returned value 0 indicates that the mic is off, while 1 is on.
API prototype
Future<int> GetMicState()
Sample code
int micState = await ITMGContext.GetInstance().GetAudioCtrl().GetMicState();
Enabling or disabling capturing device
This API is used to enable/disable a capturing device. The device is not enabled by default after room entry.
This API can only be called after room entry. The device will be disabled automatically after room exit.
Operations such as permission application and volume type adjustment will generally be performed when a capturing device is enabled on a mobile device.
API prototype
Future<int> EnableAudioCaptureDevice(bool enable)
|
enable | bool | To enable the capturing device, set this parameter to true, otherwise, set it to false. |
Sample code
// Enable capturing device
int res = await ITMGContext.GetInstance().GetAudioCtrl().EnableAudioCaptureDevice(true);
Getting the capturing device status
This API is used to get the status of a capturing device.
API prototype
Future<bool> IsAudioCaptureDeviceEnabled()
Sample code
bool res = await ITMGContext.GetInstance().GetAudioCtrl().IsAudioCaptureDeviceEnabled();
Enabling or disabling audio upstreaming
This API is used to enable/disable audio upstreaming. If a capturing device is already enabled, it will send captured audio data; otherwise, it will remain mute. For more information on how to enable/disable the capturing device, see the EnableAudioCaptureDevice API.
API prototype
Future<int> EnableAudioSend(bool enable)
|
isEnabled | bool | To enable audio upstreaming, set this parameter to true; otherwise, set it to false. |
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().EnableAudioSend(isCheck);
Getting audio upstreaming status
This API is used to get the status of audio upstreaming.
API prototype
Future<bool> IsAudioSendEnabled()
Sample code
bool IsAudioSend = await ITMGContext.GetInstance().GetAudioCtrl().IsAudioSendEnabled();
Getting the real-time mic volume
This API is used to get the real-time mic volume level. A number-type value in the range of 0–100 will be returned. We recommend that you call this API once every 20 ms.
API prototype
Future<int> GetMicLevel()
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().GetMicLevel();
Getting the real-time audio upstreaming volume
This API is used to get the local real-time audio upstreaming volume level. A number-type value in the range of 0–100 will be returned.
API prototype
Future<int> GetSendStreamLevel()
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().GetSendStreamLevel();
Setting the mic software volume
This API is used to set the mic volume level. The corresponding parameter is volume, which is equivalent to attenuating or gaining the captured sound.
API prototype
Future<int> SetMicVolume(int volume)
|
volume | number | Value range: 0–200. Default value: `100`. `0` indicates that the audio is mute, while `100` indicates that the volume level remains unchanged. |
Sample code
int volume = 100;
int res = await ITMGContext.GetInstance().GetAudioCtrl().SetMicVolume(volume);
Getting the mic software volume
This API is used to get the mic volume level. A number-type value will be returned. 101 indicates that the SetMicVolume API has not been called.
API prototype
Future<int> GetMicVolume()
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().GetMicVolume();
Voice Chat Playback APIs
|
EnableSpeaker | Enables/Disables the speaker. |
GetSpeakerState | Gets the speaker status. |
EnableAudioPlayDevice | Enables/Disables the playback device. |
IsAudioPlayDeviceEnabled | Gets the playback device status. |
EnableAudioRecv | Enables/Disables audio downstreaming. |
IsAudioRecvEnabled | Gets the audio downstreaming status. |
GetSpeakerLevel | Gets the real-time speaker volume level. |
GetRecvStreamLevel | Gets the real-time downstreaming audio levels of other members in the room. |
SetSpeakerVolume | Sets the speaker volume level. |
GetSpeakerVolume | Gets the speaker volume level. |
Enabling or disabling speaker
This API is used to enable/disable the speaker. EnableSpeaker = EnableAudioPlayDevice + EnableAudioRecv
API prototype
Future<int> EnableSpeaker(bool enable)
|
bEnable | bool | To disable the speaker, set this parameter to `false`; otherwise, set it to `true`. |
Sample code
// Turn on the speaker
await ITMGContext.GetInstance().GetAudioCtrl().EnableSpeaker(isCheck);
Getting the speaker status
This API is used to get the speaker status. 0 indicates that the speaker is off, and 1 is on.
API prototype
Future<int> GetSpeakerState()
Sample code
int spkState = await ITMGContext.GetInstance().GetAudioCtrl().GetSpeakerState();
Enabling or disabling playback device
This API is used to enable/disable a playback device.
API prototype
Future<int> EnableAudioPlayDevice(bool enable)
|
enable | bool | To disable the playback device, set this parameter to `false`; otherwise, set it to `true`. |
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().EnableAudioPlayDevice(isCheck);
Getting the playback device status
This API is used to get the status of a playback device.
API prototype
Future<bool> IsAudioPlayDeviceEnabled()
Sample code
bool res = await ITMGContext.GetInstance().GetAudioCtrl().IsAudioPlayDeviceEnabled();
Enabling or disabling audio downstreaming
This API is used to enable/disable audio downstreaming. If a playback device is already enabled, it will play back audio data from other members in the room; otherwise, it will remain mute. For more information on how to enable/disable the playback device, see the EnableAudioPlayDevice API.
API prototype
Future<int> EnableAudioRecv(bool enable)
|
isEnabled | bool | To enable audio downstreaming, set this parameter to `true`; otherwise, set it to `false`. |
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().EnableAudioRecv(isCheck);
Getting audio downstreaming status
This API is used to get the status of audio downstreaming.
API prototype
Future<bool> IsAudioRecvEnabled()
Sample code
bool res = await ITMGContext.GetInstance().GetAudioCtrl().IsAudioRecvEnabled();
Getting the real-time speaker volume
This API is used to get the real-time speaker volume level. A number-type value will be returned to indicate the volume level. We recommend that you call this API once every 20 ms.
API prototype
Future<int> GetSpeakerLevel()
Sample code
bool res = await ITMGContext.GetInstance().GetAudioCtrl().GetSpeakerLevel();
Getting the real-time downstreaming audio levels of other members in room
This API is used to get the real-time audio downstreaming volume of other members in the room. A number-type value will be returned. Value range: 0–200.
API prototype
Future<int> GetRecvStreamLevel(String openID)
|
openId | string | `openId` of another member in the room |
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().GetRecvStreamLevel(_editRoomManagerID.text);
Dynamically setting the volume of a member of the room
This API is used to set the volume of a member in the room. It takes effect only on the local.
API prototype
Future<int> SetSpeakerVolumeByOpenID(String openId, int volume)
|
openId | string | `OpenID` of the target user |
volume | number | Percentage. Recommended value range: 0-200. Default value: `100`. |
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().SetSpeakerVolumeByOpenID(_editRoomManagerID.text, 100);
Getting volume percentage
This API is used to get the volume level set by SetSpeakerVolumeByOpenID.
API prototype
Future<int> GetSpeakerVolumeByOpenID(String openId)
|
openId | string | `OpenID` of the target user |
Returned values
API returns volume percentage set by OpenID, where 100 is by default.
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().GetSpeakerVolumeByOpenID(_editRoomManagerID.text);
Setting the speaker volume
This API is used to set the speaker volume.
API prototype
Future<int> SetSpeakerVolume(int volume)
|
volume | number | Volume level. Value range: 0-200. Default value: `100`. `0` indicates that the audio is mute, while `100` indicates that the volume level remains unchanged. |
Sample code
int volume = value.toInt();
int res = await ITMGContext.GetInstance().GetAudioCtrl().SetSpeakerVolume(volume);
Getting the speaker volume
This API is used to get the speaker volume. A number-type value will be returned to indicate the volume. 101 indicates that the SetSpeakerVolume API has not been called.\n"Level" indicates the real-time volume, and "Volume" the speaker volume. The final volume = Level * Volume%. For example, if the "Level" is 100 and "Volume" is 60, the final volume is "60".
API prototype
Future<int> GetSpeakerVolume()
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().GetSpeakerVolume();
Advanced APIs
Enabling in-ear monitoring
This API is used to enable in-ear monitoring. You need to call EnableLoopBack+EnableSpeaker before you can hear your own voice.
API prototype
Future<int> EnableLoopBack(bool enable)
|
enable | bool | Specifies whether to enable. |
Sample code
int res = await ITMGContext.GetInstance().GetAudioCtrl().EnableLoopBack(true);
Getting user's room audio type
This API is used to get a user's room audio type. The returned value is the room audio type. Value 0 indicates that an error occurred while getting the user's room audio type. For room audio types, see the EnterRoom API.
API prototype
Future<int> GetRoomType()
Sample code
int curType = await ITMGContext.GetInstance().GetRoom().GetRoomType();
Changing the room type
This API is used to modify a user's room audio type. For the result, see the callback event. The event type is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE. The audio type of the room is determined by the first user to enter the room. After that, if a member in the room changes the room type, it will take effect for all members there.
API prototype
Future<int> ChangeRoomType(int roomType)
|
roomtype | number | Room type to be switched to. For room audio types, see the `EnterRoom` API. |
Sample code
int res = await ITMGContext.GetInstance().GetRoom().ChangeRoomType(1);
Callback event
After the room type is set, the event message ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE will be returned in the callback. The returned parameters include result, error_info, and new_room_type. The new_room_type represents the following information. The event message will be identified in the OnEvent function.
|
ITMG_ROOM_CHANGE_EVENT_ENTERROOM | 1 | Indicates that the existing audio type is inconsistent with and changed to that of the entered room. |
ITMG_ROOM_CHANGE_EVENT_START | 2 | Indicates that a user is already in the room and the audio type starts changing (e.g., calling the `ChangeRoomType` API to change the audio type). |
ITMG_ROOM_CHANGE_EVENT_COMPLETE | 3 | Indicates that a user is already in the room and the audio type has been changed. |
ITMG_ROOM_CHANGE_EVENT_REQUEST | 4 | Indicates that a room member calls the `ChangeRoomType` API to request a change of room audio type. |
Sample code
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE:
{
// Process room type events
}
break;
The monitoring event of room call quality
This is the quality monitoring event used to listen on the network quality. If your network conditions are poor, the business layer will ask you to switch the network through the UI. This event is triggered once every two seconds after room entry, and its message is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY. The returned parameters include weight, loss, and delay, which are as detailed below:
|
weight | number | Value range: 1–50. `50` indicates excellent sound quality, `1` indicates very poor (barely usable) sound quality, and `0` represents an initial meaningless value. Generally, if the value is below 30, you can remind users that the network is poor and recommend them to switch the network. |
loss | var | Upstream packet loss rate |
delay | number | Voice chat delay in ms |
Getting version number
This API is used to get the SDK version number for analysis.
API prototype
Future<String> GetSDKVersion()
Sample code
_sdkVersions = await ITMGContext.GetInstance().GetSDKVersion();
Setting the application name and version
This API is used to set the application name and version.
API prototype
Future<void> SetAppVersion(String appVersion)
Parameter description
|
appVersion | string | Application name and version |
Sample code
await ITMGContext.GetInstance().SetAppVersion("gme V2.0.0");
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
Future<int> SetLogLevel(int levelWrite, int levelPrint)
Parameter description
|
level | number | Sets the log level. `TMG_LOG_LEVEL_NONE` indicates not to log. Default value: `TMG_LOG_LEVEL_INFO`. |
level description:
|
TMG_LOG_LEVEL_NONE | Does not print logs |
TMG_LOG_LEVEL_ERROR | Prints error logs (default) |
TMG_LOG_LEVEL_INFO | Prints info logs |
TMG_LOG_LEVEL_DEBUG | Prints debug logs |
TMG_LOG_LEVEL_VERBOSE | Prints verbose logs |
Sample code
ITMGContext.GetInstance().SetLogLevel(ITMG_LOG_LEVEL.TMG_LOG_LEVEL_ERROR, ITMG_LOG_LEVEL.TMG_LOG_LEVEL_ERROR);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
API prototype
Future<int> SetLogPath(String logDir)
Sample code
String curPath = ""// Set a path by yourself
ITMGContext.GetInstance().SetLogPath(curPath);
Getting the diagnostic messages
This API is used to get information on the quality of real-time audio/video calls, which is mainly used to view real-time call quality and troubleshoot and can be ignored on the business side.
API prototype
Future<String> GetQualityTips()
Sample code
String curQualityTips = await ITMGContext.GetInstance().GetRoom().GetQualityTips();
Callback message
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | A member entered the audio room. | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | A member exited the audio room. | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | The room was disconnected for network or other reasons. | result; error_info | {"error_info":"waiting timeout, please check your network","result":0} |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | Room members were updated. | user_list; event_id | {"event_id":1,"user_list":["0"]} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_START | The reconnection to the room started. | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS | The reconnection to the room succeeded. | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM | The room was quickly switched. | result; error_info | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | The room status was changed. | result; error_info; sub_event_type; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_START | Cross-room mic connect started. | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_STOP | Cross-room mic connect stopped. | result; | {"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_DEFAULT_DEVICE_CHANGED | The default speaker device was changed. | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_NEW_DEVICE | A new speaker device was added. | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_LOST_DEVICE | A speaker device was lost. | result; error_info | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_NEW_DEVICE | A new mic device was added. | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_LOST_DEVICE | A mic device was lost. | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_DEFAULT_DEVICE_CHANGED | The default mic device was changed. | result; error_info | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | The room quality changed. | weight; loss; delay | {"weight":5,"loss":0.1,"delay":1} |
ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE | Voice message recording was completed. | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE | Voice message upload was completed. | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE | Voice message download was completed. | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE | Voice message playback was completed. | result; file_path | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE | Fast speech-to-text conversion was completed. | result; text;file_id | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE | Streaming speech-to-text conversion was completed. | result; file_path; text;file_id | {{"file_id":"","file_path":","text":"","result":0}} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING | Streaming speech-to-text conversion is in progress. | result; file_path; text;file_id | {{"file_id":"","file_path":","text":"","result":0}} |
ITMG_MAIN_EVNET_TYPE_PTT_TEXT2SPEECH_COMPLETE | Text-to-speech conversion was completed. | result; text;file_id | {{"file_id":"","text":"","result":0}} |
ITMG_MAIN_EVNET_TYPE_PTT_TRANSLATE_TEXT_COMPLETE | Text translation was completed. | result; text;file_id | {{"file_id":"","text":"","result":0}} |
This document describes how to integrate with and debug GME client APIs for the voice messaging and speech-to-text services for Flutter.
Key Considerations for Using GME
GME provides the real-time voice service and voice messaging and speech-to-text services, which all depend on core APIs such as Init and Poll.
Notes
You have created a GME application and obtained the SDK AppID and key. For more information, see Activating Services. You have activated GME real-time voice and voice messaging and speech-to-text services. For more information, see Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, GmeError.AV_OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Note: \nThere is a default call rate limit for speech-to-text APIs. For more information on how calls are billed within the limit, see Purchase Guide. If you want to increase the limit or learn more about how excessive calls are billed, submit a ticket. Non-streaming speech-to-text API SpeechToText(): There can be up to 10 concurrent requests per account.
Streaming speech-to-text API StartRecordingWithStreamingRecognition(): There can be up to 50 concurrent requests per account.
Integrating the SDK
Directions
Key processes involved in SDK integration are as follows:
.dart files
gme.dart GME business implementation APIs
gmeType.dart GME type definition file
gmeError.dart GME error type definition file
Core APIs
|
InitSDK | Initializes GME. |
Poll | Triggers the event callback. |
Uninit | Uninitializes GME. |
Importing the GME module
import 'package:gme/gme.dart';
import 'package:gme/gmeType.dart';
Getting an instance
var m_context = await ITMGContext.GetInstance();
Initializing the SDK
You need to initialize the SDK through the Init API before you can use the real-time voice chat, voice messaging, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
Future<int> InitSDK(String appID, String openID)
|
sdkAppId | string | |
openID | string | `openID` can only be in `Int64` type, which is passed in after being converted to a string. You can customize its rules, and it must be unique in the application. To pass in `openID` as a string, submit a ticket for application. |
Returned values
|
GmeError.AV_OK= 0 | SDK initialized successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Solution: Check whether the SDK file is complete. We recommend that you delete it and then import the SDK again. |
Note:
Notes on 7015 error code
The 7015 error code is identified by MD5. If this error is reported during integration, check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Sample code
int res = await ITMGContext.GetInstance().InitSDK(_editAppID.text,_editOpenID.text);
// Determine whether the initialization is successful by the returned value
if (ret != GmeError.AV_OK)
{
print("Failed to initialize the SDK:");
return;
}
Triggering event callback
Event callbacks can be triggered by calling the Poll API in the timer. The Poll API is GME's message pump and should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run abnormally. For more information, see the EnginePollHelper file in SDK Download Guide. Note:
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
Sample code
Future<void> pollTimer() async {
_pollTimer = Timer.periodic(Duration(milliseconds: 100), (Timer timer) {
ITMGContext.GetInstance().Poll();
});
}
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Messaging and Speech-to-Text Services
Note:
The speech-to-text service consists of fast recording-to-text conversion and streaming speech-to-text conversion.
You do not need to enter a voice chat room when using the voice messaging service.
The maximum recording duration of a voice message is 58 seconds by default, and the minimum recording duration cannot be less than 1 second. If you want to customize the recording duration, for example, to change the maximum recording duration to 10 seconds, call the SetMaxMessageLength API to set it after initialization.
Flowchart for using the voice message service
Flowchart for using the speech-to-text service
|
GenAuthBuffer | Gets the authentication information. |
SetMaxMessageLength | Specifies the maximum duration of a voice message. |
Generating the local authentication key
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, use the backend deployment key as detailed in Authentication Key. API prototype
Future<Uint8List> GenAuthBuffer(String appID, String roomID, String openID, String key)
|
appId | string | `AppId` from the Tencent Cloud console |
roomId | string | Enter `null` or an empty string. |
openId | string | User ID, which is the same as `OpenId` during initialization. |
key | string | Permission key from the Tencent Cloud console. |
Application authentication
After the authentication information is generated, the authentication is assigned to the SDK.
API prototype
Future<int> ApplyPTTAuthbuffer(Uint8List authBuffer)
Sample code
Uint8List authBuffer = await ITMGContext.GetInstance().GenAuthBuffer(_editAppID.text, _editRoomID.text, _editOpenID.text, _editKey.text);
m_context.ApplyPTTAuthbuffer(authBuffer);
Specifying the maximum duration of voice message
This API is used to specify the maximum duration of a voice message, which can be up to 58 seconds.
API prototype
Future<int> SetMaxMessageLength(int msTime)
|
msTime | number | Voice message duration in ms. Value range: 1000 < `msTime` <= 58000 |
Sample code
ITMGContext.GetInstance().GetPTT().SetMaxMessageLength(fileLen);
Streaming Speech Recognition
Voice messaging and speech-to-text APIs
|
StartRecordingWithStreamingRecognition | Starts streaming recording. |
StopRecording | Stops recording. |
Starting streaming speech recognition
This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. It can specify a language for recognition or translate the text recognized in speech into a specified language and return the translation. To stop recording, call StopRecording. API prototype
Future<int> StartRecordingWithStreamingRecognition(String filePath, String speechLanguage, String translateLanguage)
|
filePath | string | Path of the stored audio file |
speechLanguage | string | |
translateLanguage | string | |
Sample code
string filePath = "xx/xxx/xxx.silk"
int res = await ITMGContext.GetInstance().GetPTT().StartRecordingWithStreamingRecognition(filePath, strCurLanguage, strCurLanguage);
if (ret == 0) {
this.currentStatus = "Start streaming recording";
} else {
this.currentStatus = "Failed to start streaming recording";
}
Note:
Translation incurs additional fees. For more information, see Purchase Guide. Callback for streaming speech recognition
After streaming speech recognition is started, you need to listen on callback messages in the OnEvent notification, which is as detailed below:
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.\nITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING returns the recognized text in real time during the recording, which is equivalent to returning the recognized text while speaking.
The event message will be identified in the callback notification based on the actual needs. The passed parameters include the following four messages.
|
result | Return code indicating whether streaming speech recognition is successful |
text | Text converted from speech |
file_path | Local path of the stored recording file |
file_id | Backend URL address of the recording file, which will be retained for 90 days |
Note:
The file_id is empty when the 'ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING' message is listened for.
Error codes
|
32775 | Streaming speech-to-text conversion failed, but recording succeeded. | Call the `UploadRecordedFile` API to upload the recording file and then call the `SpeechToText` API to perform speech-to-text conversion. |
32777 | Streaming speech-to-text conversion failed, but recording and upload succeeded. | The message returned contains a backend URL after successful upload. Call the `SpeechToText` API to perform speech-to-text conversion. |
32786 | Streaming speech-to-text conversion failed. | During streaming recording, wait for the execution result of the streaming recording API to return. |
32787 | Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
32788 | Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
void handleEventMsg(int eventType, String data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE:
{
HandleSTREAM2TEXTComplete(data,true);
break;
}
...
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING:
{
HandleSTREAM2TEXTComplete(data, false);
break;
}
}
}
ITMGContext.GetInstance().SetEvent(handleEventMsg);
Voice Message Recording
The recording process is as follows: start recording > stop recording > return recording callback > start the next recording.
Voice messaging and speech-to-text APIs
|
StartRecording | Starts recording. |
PauseRecording | Pauses recording. |
ResumeRecording | Resumes recording. |
StopRecording | Stops recording. |
CancelRecording | Cancels recording. |
Starting recording
This API is used to start recording.
API prototype
Future<int> StartRecording(String filePath)
|
filePath | string | Path of the stored voice message file |
Sample code
string filepath = "xxxx/xxx.silk";
int res = await ITMGContext.GetInstance().GetPTT().StartRecording(filepath);
Stopping recording
This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.
API prototype
Future<int> StopRecording()
Sample code
ITMGContext.GetInstance().GetPTT().StopRecording();
Callback for recording start
A callback will be executed through a delegate function to pass a message when recording is completed.
To stop recording, call StopRecording. The callback for recording start will be returned after the recording is stopped.
|
code | string | `0`: Recording is completed. |
filepath | string | Path of the stored recording file, which must be accessible and cannot be the `fileid`. |
Error codes
|
4097 | A parameter is empty. | Check whether the API parameters in the code are correct. |
4098 | An initialization error occurred. | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
4099 | Recording is in progress. | Make sure that the SDK recording feature is used at the right time. |
4100 | No audio data is captured. | Check whether the mic is working properly. |
4101 | An error occurred while accessing the file during recording. | Ensure the existence of the file and the validity of the file path. |
4102 | The mic is not authorized. | The mic permission is required for using the SDK. To add the permission, see the SDK project configuration document for the corresponding engine or platform. |
4103 | The recording duration is too short. | The recording duration should be in ms and longer than 1,000 ms. |
4104 | No recording operation is started. | Check whether the recording starting API has been called. |
Sample code
void handleEventMsg(int eventType, String data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE:
{
// Process
break;
}
}
}
ITMGContext.GetInstance().SetEvent(handleEventMsg);
Pausing recording
This API is used to pause recording. If you want to resume recording, call the ResumeRecording API.
API prototype
Future<int> PauseRecording()
Sample code
ITMGContext.GetInstance().GetPTT().PauseRecording();
Resuming recording
This API is used to resume recording.
API prototype
Future<int> ResumeRecording()
Sample code
ITMGContext.GetInstance().GetPTT().ResumeRecording();
Canceling recording
This API is used to cancel recording. There is no callback after cancellation.
API prototype
Future<int> CancelRecording()
Sample code
ITMGContext.GetInstance().GetPTT().CancelRecording();
Voice Message Upload, Download, and Playback
|
UploadRecordedFile | Uploads an audio file. |
DownloadRecordedFile | Downloads an audio file. |
PlayRecordedFile | Plays back audio. |
StopPlayFile | Stops playing back audio. |
GetFileSize | Gets the audio file size. |
GetVoiceFileDuration | Gets the audio file duration. |
Uploading an audio file
This API is used to upload an audio file.
API prototype
Future<int> UploadRecordedFile(String filePath)
|
filePath | String | Path of the uploaded audio file, which is a local path. |
Sample code
ITMGContext.GetInstance().GetPTT().UploadRecordedFile(_filePath);
Callback for audio file upload completion
After the audio file is uploaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
result | number | `0`: Recording is completed. |
filepath | string | Path of the stored recording file |
fileid | string | File URL |
Error codes
|
8193 | An error occurred while accessing the file during upload. | Ensure the existence of the file and the validity of the file path. |
8194 | Signature verification failed. | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
8195 | A network error occurred. | Check whether the device can access the internet. |
8196 | The network failed while getting the upload parameters. | Check whether the authentication is correct and whether the device network can normally access the internet. |
8197 | The packet returned during the process of getting the upload parameters is empty. | Check whether the authentication is correct and whether the device network can normally access the internet. |
8198 | Failed to decode the packet returned during the process of getting the upload parameters. | Check whether the authentication is correct and whether the device network can normally access the internet. |
8200 | No `appinfo` is set. | Check whether the `apply` API is called or whether the input parameters are empty. |
Sample code
void handleEventMsg(int eventType, String data){
switch (eventType) {
...
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE:
{
// Process
break;
}
}
}
ITMGContext.GetInstance().SetEvent(handleEventMsg);
Downloading the audio file
This API is used to download an audio file.
API prototype
Future<int> DownloadRecordedFile(String fileId, String filePath)
|
fileId | string | File URL |
filePath | string | Local path of the saved file, which must be accessible and cannot be the `fileid`. |
Sample code
ITMGContext.GetInstance().GetPTT().DownloadRecordedFile(_fileId, _filePath);
Callback for audio file download completion
After the audio file is downloaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
result | number | `0`: Download is completed. |
filepath | string | Path of the stored recording file |
fileid | string | URL of the recording file, which will be retained on the server for 90 days. |
Error codes
|
12289 | An error occurred while accessing the file during download. | Check whether the file path is valid. |
12290 | Signature verification failed. | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
12291 | A network storage system exception occurred. | The server failed to get the audio file. Check whether the API parameter `fileid` is correct, whether the network is normal, and whether the file exists in COS. |
12292 | A server file system error occurred. | Check whether the device can access the internet and whether the file exists on the server. |
12293 | The HTTP network failed while getting the download parameters. | Check whether the device can access the internet. |
12294 | The packet returned during the process of getting the download parameters is empty. | Check whether the device can access the internet. |
12295 | Failed to decode the packet returned during the process of getting the download parameters. | Check whether the device can access the internet. |
12297 | No `appinfo` is set. | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
Sample code
void handleEventMsg(int eventType, String data){
switch (eventType) {
...
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
}
}
ITMGContext.GetInstance().SetEvent(handleEventMsg);
Playing back audio
This API is used to play back audio.
API prototype
Future<int> PlayRecordedFile(String filePath, int voiceType)
|
filePath | string | Local audio file path |
voicetype | ITMG_VOICE_TYPE | |
Error codes
|
20485 | Playback is not started. | Ensure the existence of the file and the validity of the file path. |
Sample code
int res = await ITMGContext.GetInstance().GetPTT().PlayRecordedFile(_filePath, _nVoiceType);
Callback for audio playback
After the audio is played back, the event message ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameter includes result and file_path.
|
code | number | `0`: Playback is completed. |
filepath | string | Path of the stored recording file |
Error codes
|
20481 | An initialization error occurred. | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
20482 | During playback, the client tried to interrupt and play back the next one but failed (which should succeed normally). | Check whether the code logic is correct. |
20483 | A parameter is empty. | Check whether the API parameters in the code are correct. |
20484 | An internal error occurred. | An error occurred while initializing the player. This error code is generally caused by failure in decoding, and the error should be located with the aid of logs. |
Sample code
void handleEventMsg(int eventType, String data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE:
{
// Process
break;
}
}
}
Stopping audio playback
This API is used to stop audio playback. There will be a callback for playback completion when the playback stops.
API prototype
Future<int> StopPlayFile()
Sample code
ITMGContext.GetInstance().GetPTT().StopPlayFile();
Getting audio file size
This API is used to get the size of an audio file.
API prototype
Future<int> GetFileSize(String filePath)
|
filePath | string | Path of the audio file, which is a local path |
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().GetFileSize(_filePath);
Getting audio file duration
This API is used to get the duration of an audio file in milliseconds.
API prototype
Future<int> GetVoiceFileDuration(String filePath)
|
filePath | string | Path of the audio file, which is a local path |
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().GetVoiceFileDuration(_filePath);
Fast Recording-to-Text Conversion
Translating audio file into text in specified language
This API can specify a language for recognition or translate the text recognized in speech into a specified language and return the translation.
Note：
Translation incurs additional fees. For more information, see Purchase Guide. API prototype
Future<int> SpeechToText(String fileId, String speechLanguage, String translateLanguage)
|
fileID | string | URL of the audio file, which will be retained on the server for 90 days. |
speechLanguage | string | |
translatelanguage | string | |
Sample code
ITMGContext.GetInstance().GetPTT().SpeechToText(_fileId, "cmn-Hans-CN", "cmn-Hans-CN");
Callback for recognition
After the specified audio file is converted to text, the event message ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path and text (recognized text).
|
result | number | `0`: Recording is completed. |
fileid | string | URL of the audio file, which will be retained on the server for 90 days. |
text | string | Converted text |
Error codes
|
32769 | An internal error occurred. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
32770 | A network error occurred. | Check whether the device can access the internet. |
32772 | Failed to decode the returned packet. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
32774 | No `appinfo` is set. | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
32776 | `authbuffer` check failed. | Check whether `authbuffer` is correct. |
32784 | The speech-to-text conversion parameter is incorrect. | Check whether the API parameter `fileid` in the code is empty. |
32785 | Speech-to-text translation returned an error. | An error occurred in the voice messaging and speech-to-text feature on the backend. Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
32787 | Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
32788 | Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
void handleEventMsg(int eventType, String data){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE:
{
// Process
break;
}
}
}
Voice Message Volume Level APIs
|
GetMicLevel | Gets the real-time mic volume level. |
SetMicVolume | Sets the recording volume level. |
GetMicVolume | Gets the recording volume level. |
GetSpeakerLevel | Gets the real-time speaker volume level. |
SetSpeakerVolume | Sets the playback volume level. |
GetSpeakerVolume | Gets the playback volume level. |
Getting the real-time mic volume of voice message
This API is used to get the real-time mic volume. A number-type value will be returned. Value range: 0-200.
API prototype
Future<int> GetMicLevel()
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().GetMicLevel();
Setting the recording volume of voice message
This API is used to set the recording volume of voice message. Value range: 0-200.
API prototype
Future<int> SetMicVolume(int volume)
|
vol | number | Value range: 0-200. Default value: `100`. `0` indicates that the audio is mute, while `100` indicates that the volume level remains unchanged. |
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().SetMicVolume(100);
Getting the recording volume of voice message
This API is used to get the recording volume of a voice message. A number-type value will be returned. Value range: 0-200.
API prototype
Future<int> GetMicVolume()
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().GetMicVolume();
Getting the real-time speaker volume of voice message
This API is used to get the real-time speaker volume. A number-type value will be returned. Value range: 0-200.
API prototype
Future<int> GetSpeakerLevel()
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().GetSpeakerLevel();
Setting the playback volume of voice message
This API is used to set the playback volume of voice message. Value range: 0-200.
API prototype
Future<int> SetSpeakerVolume(int volume)
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().SetSpeakerVolume(100);
Getting the playback volume of voice message
This API is used to get the playback volume of a voice message. A number-type value will be returned. Value range: 0-200.
API prototype
Future<int> GetSpeakerVolume()
Sample code
final int res = await ITMGContext.GetInstance().GetPTT().GetSpeakerVolume();
Advanced APIs
Getting version number
This API is used to get the SDK version number for analysis.
API prototype
Future<String> GetSDKVersion()
Sample code
_sdkVersions = await ITMGContext.GetInstance().GetSDKVersion();
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
Future<int> SetLogLevel(int levelWrite, int levelPrint)
Parameter description
|
level | ITMG_LOG_LEVEL | Sets the log level. `TMG_LOG_LEVEL_NONE` indicates not to log. Default value: `TMG_LOG_LEVEL_INFO`. |
level description:
|
TMG_LOG_LEVEL_NONE | Does not print logs |
TMG_LOG_LEVEL_ERROR | Prints error logs (default) |
TMG_LOG_LEVEL_INFO | Prints info logs |
TMG_LOG_LEVEL_DEBUG | Prints debug logs |
TMG_LOG_LEVEL_VERBOSE | Prints verbose logs |
Sample code
ITMGContext.GetInstance().SetLogLevel(ITMG_LOG_LEVEL.TMG_LOG_LEVEL_ERROR,ITMG_LOG_LEVEL.TMG_LOG_LEVEL_ERROR);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
API prototype
Future<int> SetLogPath(String logDir)
Sample code
String logDir = ""// Set a path by yourself
ITMGContext.GetInstance().SetLogPath(curPath);
This document describes how to configure an Electron project for the GME APIs for Electron.
Windows
Importing the SDK
Step 1: Install Node.js
1. Download the latest version of the Node.js installation package Windows Installer (.msi) 64-bit for Windows.
2. Open Node.js command prompt in the application list and open a terminal window.
Step 2. Install Electron
Run the following command in the terminal to install Electron. V4.0.0 or later is recommended.
$ npm install electron -g
Step 3. Install the GME SDK for Electron
1. Use the following npm command in your Electron project to install the GME SDK:
$ npm install gme-electron-sdk@latest --save
2. In the project script, import and use the module:
const { GmeContext } = require('gme-electron-sdk');
// import gmesdk from 'gme-electron-sdk';
gmeContext = new GmeContext();
// Get the SDK version number
gmeContext.GetSDKVersion();
Step 4. Create an executable program
Install the packaging tool. We recommend you use Electron Forge. You can run the following command:
1. Add Electron Forge to your application's development dependencies and run the import command to set the scaffold of Forge:
npm install --save-dev @electron-forge/cli
npx electron-forge import
2. Run the make command of Forge to create a distributable application:
This document describes how to integrate with and debug GME client APIs for the voice chat feature for Electron.
Key Considerations for Using GME
GME provides the voice chat, voice messaging and speech-to-text services, which all depend on core APIs such as Init and Poll.
Notes
You have created a GME application and obtained the SDK AppID and key. For more information, see Activating Services. You have activated GME voice chat, voice messaging, and speech-to-text services. For more information, see Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, GmeError.AV_OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Integrating the SDK
Directions
Key processes involved in SDK integration are as follows:
2. Calling Poll periodically to trigger callbacks Core APIs
|
| |
| Triggers an event callback. |
| |
| |
| |
Importing the GME module
const { GmeContext } = require('gme-electron-sdk');
Getting an instance
To use the voice chat feature, get the GmeSDK object first.
context = new GmeContext();
Initializing the SDK
You need to initialize the SDK through the Init API before you can use the real-time voice chat, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
//class GmeSDK
Init(appid: string, openid: string): number;
|
| | |
| | openID can only be in Int64 type, which is passed in after being converted to a string. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
Returned values
|
| The SDK was initialized successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Check whether the SDK file is complete. We recommend that you delete it and then import it again. |
Note on 7015 error code
The error code 7015 is identified based on the MD5 value. If this error is reported during integration, check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Due to the third-party enhancement, the Unity packaging mechanism, and other factors, the MD5 value of the library file will be affected, resulting in misjudgment. Therefore, ignore this error in the logic for official releases, and avoid displaying it on the UI.
Sample code
string SDKAPPID3RD = "14000xxxxx";
string openId="10001";
number ret = context.Init(SDKAPPID3RD, openId);
// Determine whether the initialization is successful by the returned value
if (ret != GmeError.AV_OK)
{
console.log("Failed to initialize the SDK:");
return;
}
Setting callbacks
The API class uses the Delegate method to send callback notifications to the application. Register the callback function to the SDK for receiving callback messages before room entry.
Function prototype and sample code
Register the callback function to the SDK for receiving callback messages before room entry.
SetTMGDelegate(cb: ITMGDelegate);
// When initializing the SDK
context = GmeSDK.GetInstance();
context.setTMGDelegate(function(eventId, msg){
if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
{
// Processing callbacks
}
});
Triggering event callback
You need to periodically call the Poll API to trigger event callbacks. The Poll API is GME's message pump and should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run abnormally. For more information, see the EnginePollHelper file in SDK Download Guide. Call the Poll API periodically
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
Sample code
setInterval(function () {
context.Poll();
}, 50);
Pausing the system
When a Pause event occurs in the system, the engine should also be notified for pause. For example, when the application switches to the background (OnApplicationPause, isPause=True), and you do not need the background to play back the audio in the room, please call Pause API to pause the GME service.
API prototype
Resuming the system
When a Resume event occurs in the system, the engine should also be notified for resumption. The Resume API only supports resuming voice chat.
API prototype
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Chat Room APIs
You should initialize and call the SDK to enter a room before voice chat can start.
|
| Calculates the local authentication key. |
| |
| |
| Determines whether room entry is successful. |
Local authentication key calculation
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, use the backend deployment key as detailed in Authentication Key. API prototype
GenAuthBuffer(appId: string,roomId: string, openId:string, appKey: number) :string;
|
| | AppID from the Tencent Cloud console
|
| | Room ID, which can contain up to 127 characters. |
| | User ID, which is the same as openID during initialization. |
| | Permission key from the Tencent Cloud console. |
Sample code
let userSig = context.GenAuthBuffer(this.appid, this.roomId, this.userId, this.authKey)
context.EnterRoom(this.roomId, this.roomType, userSig);
Entering a room
This API is used to enter a room with the generated authentication information. The mic and speaker are not enabled by default after room entry.
Note：
If the room entry callback result is 0, the room entry is successful. If 0 is returned from the EnterRoom API, it doesn't necessarily mean that the room entry is successful.
The audio type of the room is determined by the first user to enter the room. After that, if a member in the room changes the room type, it will take effect for all members there. For example, if the first user entering the room uses the smooth sound quality, and the second user entering the room uses the HD sound quality, the room audio type of the second user will become smooth sound quality. Only when a member in the room calls the ChangeRoomType API, the audio type of the room will be changed.
API prototype
EnterRoom(roomid: string, roomType: number, appKey: string) :number;
|
| | Room ID, which can contain up to 127 characters. |
| | Room type. We recommend that you select ITMG_ROOM_TYPE_FLUENCY for games. For more information on room audio types, see Sound Quality. |
| | |
Sample code
context.EnterRoom(roomID, ITMG_ROOM_TYPE_STANDARD, retAuthBuff);
Callback for room entry
After the user enters the room, the ITMG_MAIN_EVENT_TYPE_ENTER_ROOM event type will be called back to notify the room entry result, which can be listened on for processing. A successful callback means that the room entry is successful, and the billing starts.
Sample code
// Listen on an event:
gmeContext.setTMGDelegate(function(eventId, msg){
switch (eventId) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
}
}
});
Data details
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | | {"error_info":"waiting timeout, please check your network","result":0} |
If the network is disconnected, there will be a disconnection callback notification ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT. At this time, the SDK will automatically reconnect, and the callback is ITMG_MAIN_EVENT_TYPE_RECONNECT_START. When the reconnection is successful, there will be a callback ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS.
Error codes
|
| Authentication failure causes: AppID doesn't exist or is incorrect.
An error occurred while authenticating authbuff. Authentication expired. OpenId is invalid.
|
| The user was already in another room. |
| The user was already in the process of entering a room but repeated this operation. We recommend that you not call the room entry API until the room entry callback is returned. |
| The user was already in the room and called the room entry API again. |
| Make sure that the SDK is initialized, OpenId complies with the rules, the APIs are called in the same thread, and the Poll API is called normally. |
Exiting a room
This API is used to exit the current room. It is an async API. The returned value AV_OK indicates a successful async delivery. If there is a scenario in the application where room entry is performed immediately after room exit, you don't need to wait for the RoomExitComplete callback notification from the ExitRoom API; instead, you can directly call the EnterRoom API.
API prototype
Sample code
Callback for room exit
After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM. The sample code is shown below:
Sample code
gmeContext.setTMGDelegate(function(eventId, msg){
switch (eventId) {
case ITMG_MAIN_EVENT_TYPE_EXIT_ROOM:
{
// Process
break;
}
}
});
Determining whether user has entered room
This API is used to determine whether the user has entered a room. A value in boolean type will be returned. Do not call this API during room entry.
API prototype
Sample code
Room Status Maintenance
APIs in this section are used to display speaking members and members entering or exiting the room and mute a member in the room at the business layer.
|
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | The member status changed. |
| Mutes a member in the room. |
| |
| Queries whether the user of the specified openid is muted. |
Notification events of the member room entry and speaking status
This event is used to get the user speaking in the room and display the user on the UI, and to send a notification when someone enters or exits the room.
A notification for this event will be sent only when the status changes. To get the member status in real time, cache the notification when it is received at the business layer. The event message ITMG_MAIN_EVNET_TYPE_USER_UPDATE containing event_id, count, and openIdList will be returned, which will be identified in the OnEvent notification.
Notifications of the EVENT_ID_ENDPOINT_NO_AUDIO audio event will be sent only when the threshold is exceeded; that is, other members in the room can receive the notification that the local user stops speaking only after the local client captures no voice for two seconds.
The audio event returns only the member speaking status but not the specific volume level. If you need the specific volume levels of members in the room, you can use the GetVolumeById API.
|
| Return the openid of the member entering the room. | |
| Return the openid of the member exiting the room. | |
EVENT_ID_ENDPOINT_HAS_AUDIO | Return the openid of the member sending audio packets in the room. This event can be used to determine whether a user is speaking and display the voiceprint effect. | |
EVENT_ID_ENDPOINT_NO_AUDIO | Return the openid of the member stopping sending audio packets in the room. | |
Sample code
context.setTMGDelegate(function(eventId, msg){
if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
{
// Process
switch (eventID)
{
case EVENT_ID_ENDPOINT_ENTER:
// A member enters the room
break;
case EVENT_ID_ENDPOINT_EXIT:
// A member exits the room
break;
case EVENT_ID_ENDPOINT_HAS_AUDIO:
// A member sends audio packets
break;
case EVENT_ID_ENDPOINT_NO_AUDIO:
// A member stops sending audio packets
break;
default:
break;
}
break;
}
});
Muting a member in the room
This API is used to add an ID to the audio data blocklist. This operation blocks audio from someone and only applies to the local device without affecting other devices. The returned value 0 indicates that the call is successful. Assume that users A, B, and C are all speaking using their mic in a room:
If A blocks C, A can only hear B.
If B blocks neither A nor C, B can hear both of them.
If C blocks neither A nor B, C can hear both of them.
This API is suitable for scenarios where a user is muted in a room.
API prototype
AddAudioBlackList(openId: string) :number
|
| | openid of the user to be blocked
|
Sample code
context.AddAudioBlackList(openId);
Unmuting
This API is used to remove an ID from the audio data blacklist. A returned value of 0 indicates the call is successful.
API prototype
RemoveAudioBlackList(openId: string) :number
Sample code
context.RemoveAudioBlackList(openId);
Querying whether a user is muted
This API is used to query whether an ID is blocked. The returned value true indicates that the ID is blocked, while false indicates not.
API prototype
IsOpenIdInAudioBlackList(openId: string) :boolean
Sample code
boolean isInBlackList = context.IsOpenIdInAudioBlackList(openId);
Voice Chat Capturing APIs
The voice chat APIs can only be called after SDK initialization and room entry.
When the user clicks the button of enabling/disabling the mic or speaker on the UI, we recommend that you call the EnableMic or EnableSpeaker API.
To enable the user to press the mic button on the UI to speak and release it to stop speaking, we recommend that you call EnableAudioCaptureDevice once during room entry and call EnableAudioSend to enable the user to speak while pressing the button.
|
| Enables/Disables the mic. |
| |
| Enables/Disables the capturing device. |
IsAudioCaptureDeviceEnabled | Gets the capturing device status. |
| Enables/Disables audio upstreaming. |
| Gets the audio upstreaming status. |
| Gets the real-time mic volume level. |
| Gets the real-time audio upstreaming volume level. |
| Sets the mic volume level. |
| Gets the mic volume level. |
Enabling or disabling mic
This API is used to enable/disable the mic. The mic and speaker are not enabled by default after room entry. EnableMic = EnableAudioCaptureDevice + EnableAudioSend
API prototype
EnableMic(bEnable: boolean) : number
|
| | To enable the mic, set this parameter to true; otherwise, set it to false. |
Sample code
// Turn on mic
context.EnableMic(true);
Getting the mic status
This API is used to get the mic status. The returned value 0 indicates that the mic is off, while 1 is on.
API prototype
Sample code
Enabling or disabling capturing device
This API is used to enable/disable a capturing device. The device is not enabled by default after room entry.
This API can only be called after room entry. The device will be disabled automatically after room exit.
Operations such as permission application and volume type adjustment will generally be performed when a capturing device is enabled on a mobile device.
API prototype
EnableAudioCaptureDevice(enable:boolean) :number
|
| | To enable the capturing device, set this parameter to true, otherwise, set it to false. |
Sample code
// Enable capturing device
context.EnableAudioCaptureDevice(true);
Getting the capturing device status
This API is used to get the status of a capturing device.
API prototype
IsAudioCaptureDeviceEnabled():boolean
Sample code
boolean IsAudioCaptureDevice = context.IsAudioCaptureDeviceEnabled();
Enabling or disabling audio upstreaming
This API is used to enable/disable audio upstreaming. If a capturing device is already enabled, it will send captured audio data; otherwise, it will remain muted. For more information on how to enable/disable the capturing device, see the EnableAudioCaptureDevice API.
API prototype
EnableAudioSend(bEnable: boolean) :number
|
| | To enable audio upstreaming, set this parameter to true; otherwise, set it to false. |
Sample code
context.EnableAudioSend(true);
Getting audio upstreaming status
This API is used to get the status of audio upstreaming.
API prototype
IsAudioSendEnabled():boolean
Sample code
boolean IsAudioSend = context.IsAudioSendEnabled();
Getting the real-time mic volume
This API is used to get the real-time mic volume level. A number-type value in the range of 0–100 will be returned. We recommend that you call this API once every 20 ms.
API prototype
Sample code
Getting the real-time audio upstreaming volume
This API is used to get the local real-time audio upstreaming volume level. A number-type value in the range of 0–100 will be returned.
API prototype
GetSendStreamLevel() :number
Sample code
context.GetSendStreamLevel();
Setting the mic software volume
This API is used to set the mic volume level. The corresponding parameter is volume, which is equivalent to attenuating or gaining the captured sound.
API prototype
SetMicVolume(volume:number) :number
|
| | Value range: 0-200. Default value: 100. 0 indicates that the audio is muted, while 100 indicates that the volume level remains unchanged. |
Sample code
number micVol = (value * 100);
context.SetMicVolume (micVol);
Getting the mic software volume
This API is used to get the mic volume level. A number-type value will be returned. 101 indicates that the SetMicVolume API has not been called.
API prototype
Sample code
Voice Chat Playback APIs
|
| Enables/Disables the speaker. |
| |
| Enables/Disables the playback device. |
| Gets the playback device status. |
| Enables/Disables audio downstreaming. |
| Gets the audio downstreaming status. |
| Gets the real-time speaker volume level. |
| Gets the real-time downstreaming audio levels of other members in the room. |
| Sets the speaker volume level. |
| Gets the speaker volume level. |
Enabling or disabling speaker
This API is used to enable/disable the speaker. EnableSpeaker = EnableAudioPlayDevice + EnableAudioRecv
API prototype
EnableSpeaker(bEnable: boolean) : number;
|
| | To disable the speaker, set this parameter to false; otherwise, set it to true. |
Sample code
// Turn on the speaker
context.EnableSpeaker(true);
Getting the speaker status
This API is used to get the speaker status. 0 indicates that the speaker is off, and 1 is on.
API prototype
GetSpeakerState() :number
Sample code
context.GetSpeakerState();
Enabling or disabling playback device
This API is used to enable/disable a playback device.
API prototype
EnableAudioPlayDevice(enable:boolean) :number
|
| | To disable the playback device, set this parameter to false; otherwise, set it to true. |
Sample code
context.EnableAudioPlayDevice(true);
Getting the playback device status
This API is used to get the status of a playback device.
API prototype
IsAudioPlayDeviceEnabled() :boolean
Sample code
boolean enable = context.IsAudioPlayDeviceEnabled();
Enabling or disabling audio downstreaming
This API is used to enable/disable audio downstreaming. If a playback device is already enabled, it will play back audio data from other members in the room; otherwise, it will remain muted. For more information on how to enable/disable the playback device, see the EnableAudioPlayDevice API.
API prototype
EnableAudioRecv(bEnable: boolean) :number
|
| | To enable audio downstreaming, set this parameter to true; otherwise, set it to false. |
Sample code
context.EnableAudioRecv(true);
Getting audio downstreaming status
This API is used to get the status of audio downstreaming.
API prototype
IsAudioRecvEnabled():boolean
Sample code
boolean IsAudioRecv = context.IsAudioRecvEnabled();
Getting the real-time speaker volume
This API is used to get the real-time speaker volume level. A number-type value will be returned to indicate the volume level. We recommend that you call this API once every 20 ms.
API prototype
Sample code
context.GetSpeakerLevel();
Getting the real-time downstreaming audio levels of other members in room
This API is used to get the real-time audio downstreaming volume levels of other members in the room. A number-type value will be returned. Value range: 0–200.
API prototype
GetRecvStreamLevel(openId: string) :number
|
| | openId of another member in the room
|
Sample code
number level =GetRecvStreamLevel(openId);
Dynamically setting the volume of a member of the room
This API is used to set the volume of a member in the room. It takes effect only on the local.
API prototype
SetSpeakerVolumeByOpenID(openId: string, volume:number) :number;
|
| | OpenID of the target user
|
| | Percentage. Recommended value range: 0–200. Default value: 100. |
Sample code
context.SetSpeakerVolumeByOpenID(openId, 100);
Getting volume percentage
This API is used to get the volume level set by SetSpeakerVolumeByOpenID.
API prototype
GetSpeakerVolumeByOpenID(openId: string) :number;
|
| | OpenID of the target user
|
Returned values
API returns volume percentage set by OpenID, where 100 is by default.
Sample code
context.GetSpeakerVolumeByOpenID(openId);
Setting the speaker volume
This API is used to set the speaker volume.
API prototype
SetSpeakerVolume(volume:number) :number
|
| | Volume level. Value range: 0–200. Default value: 100. 0 indicates that the audio is muted, while 100 indicates that the volume level remains unchanged. |
Sample code
number vol = 100;
context.SetSpeakerVolume(vol);
Getting the speaker volume
This API is used to get the speaker volume. A number-type value will be returned to indicate the volume. 101 indicates that the SetSpeakerVolume API has not been called.\n"Level" indicates the real-time volume, and "Volume" the speaker volume. The final volume = Level * Volume%. For example, if the "Level" is 100 and "Volume" is 60, the final volume is "60".
API prototype
GetSpeakerVolume() :number
Sample code
numbet volume = context.GetSpeakerVolume();
Device Selection APIs
Device selection APIs can be used only on PC.
|
| |
| |
| Gets the number of speakers. |
| |
| |
| |
Getting the number of mics
This API is used to get the number of mics.
Function prototype
GetMicListCount() :number
Sample code
var micListCount = context.GetMicListCount();
Enumerating mics
This API is used together with the GetMicListCount API to enumerate mics.
Function prototype
GetMicList() :GmeAudioDeviceInfo[];
Sample code
var micList = context.GetMicList();
Selecting mic
This API is used to select a mic. If this API is not called or DEVICEID_DEFAULT is passed in, the default mic will be selected.\nThe 0th device id returned in the GetMicList API is the default device of the call device. If there is a selected call device, it will be maintained by service. If it is unplugged, the call device will be changed back into the default device.
Function prototype
SelectMic(micId: string) :number;
|
| | Mic ID, which is from the list returned by GetMicList. |
Sample code
context.SelectMic(deviceID);
This API is used to get the number of speakers.
Function prototype
GetSpeakerListCount() :number;
Sample code
context.GetSpeakerListCount();
Enumerating speakers
This API is used together with the GetSpeakerListCount API to enumerate speakers.
Function prototype
GetSpeakerList(): GmeAudioDeviceInfo[]
Sample code
var speakList = GetSpeakerList();
Selecting speaker
This API is used to select a playback device. If this API is not called or DEVICEID_DEFAULT is passed in, the default playback device will be selected.
Function prototype
SelectSpeaker(speakerId: string) :number
|
| | Speaker ID, which is from the list returned by GetSpeakerList. |
Sample code
var ret = SelectSpeaker(deviceID);
Advanced APIs
Enabling in-ear monitoring
This API is used to enable in-ear monitoring. You need to call EnableLoopBack+EnableSpeaker before you can hear your own voice.
API prototype
EnableLoopBack(bEnable: boolean) :number
|
| | Specifies whether to enable. |
Sample code
context.EnableLoopBack(true);
Getting user's room audio type
This API is used to get a user's room audio type. The returned value is the room audio type. Value 0 indicates that an error occurred while getting the user's room audio type. For room audio types, please see the EnterRoom API.
API prototype
Sample code
Changing the room type
This API is used to modify a user's room audio type. For the result, please see the callback event. The event type is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE. The audio type of the room is determined by the first user to enter the room. After that, if a member in the room changes the room type, it will take effect for all members there.
API prototype
ChangeRoomType(roomType: number) :number
|
| | Room type to be switched to. For room audio types, see the EnterRoom API. |
Sample code
context.ChangeRoomType(ITMG_ROOM_TYPE_FLUENCY);
Callback event
After the room type is set, the event message ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE will be returned in the callback. The returned parameters include result, error_info, and new_room_type. The new_room_type represents the following information. The event message will be identified in the OnEvent function.
|
ITMG_ROOM_CHANGE_EVENT_ENTERROOM | | The existing audio type is inconsistent with and changed to that of the entered room. |
ITMG_ROOM_CHANGE_EVENT_START | | A user is already in the room and the audio type starts changing (e.g., calling the ChangeRoomType API to change the audio type). |
ITMG_ROOM_CHANGE_EVENT_COMPLETE | | A user is already in the room, and the audio type has been changed. |
ITMG_ROOM_CHANGE_EVENT_REQUEST | | A room member calls the ChangeRoomType API to request a change of the room audio type. |
Sample code
context.setTMGDelegate(function(eventId, msg){
if (ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE == type)
{
// Process room type events
}
});
The monitoring event of room call quality
This is the quality monitoring event used to listen on the network quality. If your network conditions are poor, the business layer will ask you to switch the network through the UI. This event is triggered once every two seconds after room entry, and its message is ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY. The returned parameters include weight, loss, and delay, which are as detailed below:
|
| | Value range: 1–50. 50 indicates excellent sound quality, 1 indicates very poor (barely usable) sound quality, and 0 represents an initial meaningless value. Generally, if the value is below 30, you can remind users that the network is poor and recommend them to switch the network. |
| | Upstream packet loss rate |
| | |
Getting version number
This API is used to get the SDK version number for analysis.
API prototype
Sample code
Setting the application name and version
This API is used to set the application name and version.
API prototype
SetAppVersion(appVersion: string) : number
Parameter description
|
| | Application name and version |
Sample code
context.SetAppVersion("gme V2.0.0");
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
SetLogLevel(level: number) : number
Parameter description
|
| | Sets the log level. TMG_LOG_LEVEL_NONE indicates not to log. Default value: TMG_LOG_LEVEL_INFO. |
level description:
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
context.SetLogLevel(TMG_LOG_LEVEL_INFO);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
|
| %appdata%\\GMEGLOBAL\\GME\\ProcessName |
API prototype
SetLogPath(logPath: string)
Sample code
string logDir = ""// Set a path by yourself
context.SetLogPath(logDir);
Getting the diagnostic messages
This API is used to get information on the quality of real-time audio/video calls, which is mainly used to view real-time call quality and troubleshoot and can be ignored on the business side.
API prototype
Sample code
string tips = context.GetQualityTips();
Callback message
|
ITMG_MAIN_EVENT_TYPE_ENTER_ROOM | A member entered the audio room. | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_EXIT_ROOM | A member exited the audio room. | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_DISCONNECT | The room was disconnected due to a network or another issue. | | {"error_info":"waiting timeout, please check your network","result":0} |
ITMG_MAIN_EVNET_TYPE_USER_UPDATE | Room members were updated. | | {"event_id":1,"user_list":["0"]} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_START | The reconnection to the room started. | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_RECONNECT_SUCCESS | The reconnection to the room succeeded. | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_SWITCH_ROOM | The room was quickly switched. | | {"error_info":"","result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_TYPE | The room status was changed. | result; error_info; sub_event_type; new_room_type | {"error_info":"","new_room_type":0,"result":0} |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_START | Cross-room mic connect started. | | |
ITMG_MAIN_EVENT_TYPE_ROOM_SHARING_STOP | Cross-room mic connect stopped. | | |
ITMG_MAIN_EVENT_TYPE_SPEAKER_DEFAULT_DEVICE_CHANGED | The default speaker was changed. | | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_NEW_DEVICE | | | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_SPEAKER_LOST_DEVICE | | | {"deviceID":"{0.0.0.00000000}.{a4f1e8be-49fa-43e2-b8cf-dd00542b47ae}","deviceName":"Speaker (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":false,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_NEW_DEVICE | | | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":true,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_LOST_DEVICE | | | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_MIC_DEFAULT_DEVICE_CHANGED | The default mic was changed. | | {"deviceID":"{0.0.1.00000000}.{5fdf1a5b-f42d-4ab2-890a-7e454093f229}","deviceName":"Mic (Realtek High Definition Audio)","error_info":"","isNewDevice":false,"isUsedDevice":true,"result":0} |
ITMG_MAIN_EVENT_TYPE_CHANGE_ROOM_QUALITY | The room network quality changed. | | {"weight":5,"loss":0.1,"delay":1} |
ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE | Voice message recording was completed. | | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE | Voice message upload was completed. | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE | Voice message download was completed. | result; file_path;file_id | {"file_id":"","file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE | Voice message playback was completed. | | {"file_path":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE | Fast speech-to-text conversion was completed. | | {"file_id":"","text":"","result":0} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE | Streaming speech-to-text conversion was completed. | result; file_path; text;file_id | {{"file_id":"","file_path":","text":"","result":0}} |
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING | Streaming speech-to-text conversion is in progress. | result; file_path; text;file_id | {{"file_id":"","file_path":","text":"","result":0}} |
ITMG_MAIN_EVNET_TYPE_PTT_TEXT2SPEECH_COMPLETE | Text-to-speech conversion was completed. | | {{"file_id":"","text":"","result":0}} |
ITMG_MAIN_EVNET_TYPE_PTT_TRANSLATE_TEXT_COMPLETE | Text translation was completed. | | {{"file_id":"","text":"","result":0}} |
This document describes how to integrate with and debug GME client APIs for the voice messaging and speech-to-text services for Electron.
Key Considerations for Using GME
GME provides the real-time voice service and voice messaging and speech-to-text services, which all depend on core APIs such as Init and Poll.
Notes
You have created a GME application and obtained the SDK AppID and key. For more information, see Activating Services. You have activated GME real-time voice service and voice messaging and speech-to-text services. For more information, see Activating Services. Configure your project before using GME; otherwise, the SDK will not take effect.
After a GME API is called successfully, GmeError.AV_OK will be returned with the value being 0.
GME APIs should be called in the same thread.
The Poll API should be called periodically for GME to trigger event callbacks.
Note:
There is a default call rate limit for speech-to-text APIs. For more information on how calls are billed within the limit, see Purchase Guide. If you want to increase the limit or learn more about how excessive calls are billed, submit a ticket. Non-streaming speech-to-text API PttSpeechToText(): There can be up to 10 concurrent requests per account.
Streaming speech-to-text API PttStartRecordingWithStreamingRecognition(): There can be up to 50 concurrent requests per account.
Integrating the SDK
Directions
Key processes involved in SDK integration are as follows:
TS class
`GmeContext`: GME business implementation APIs
`GmeError`: GME error code definition class
Core APIs
Importing the GME module
const { GmeContext } = require('gme-electron-sdk');
Getting an instance
var m_context = new GmeContext();
Initializing the SDK
You need to initialize the SDK through the Init API before you can use the real-time voice, voice message, and speech-to-text services. The Init API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
API prototype
Init(appid: string, openid: string): number;
|
| | |
| | openID can only be in Int64 type, which is passed in after being converted to a string. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
Returned values
|
| Initialized the SDK successfully. |
AV_ERR_SDK_NOT_FULL_UPDATE=7015 | Checks whether the SDK file is complete. It is recommended to delete it and then import the SDK again. |
Notes on 7015 error code
The 7015 error code is judged by MD5. If this error is reported during integration, please check the integrity and version of the SDK file as prompted.
The returned value AV_ERR_SDK_NOT_FULL_UPDATE is only a reminder but will not cause an initialization failure.
Sample code
number ret = m_context.Init(sdkAppId, openID);
// Determine whether the initialization is successful by the returned value
if (ret != GmeError.AV_OK)
{
console.log("Failed to initialize the SDK:");
return;
}
Triggering event callback
Event callbacks can be triggered by calling the Poll API in the timer. The Poll API is GME's message pump and should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run abnormally. For more information, see the EnginePollHelper file in SDK Download Guide. Note:
The Poll API must be called periodically and in the main thread to avoid abnormal API callbacks.
API prototype
Sample code
setInterval(function () {
m_context.Poll();
}, 50);
Uninitializing SDK
This API is used to uninitialize the SDK to make it uninitialized. If the game business account is bound to openid, switching game account requires uninitializing GME and then using the new openid to initialize again.
API prototype
Voice Messaging and Speech-to-Text Services
Note:
The speech-to-text service consists of fast recording-to-text conversion and streaming speech-to-text conversion.
You do not need to enter a voice chat room when using the voice messaging service.
The maximum recording duration of a voice message is 58 seconds by default, and the minimum recording duration cannot be less than 1 second. If you want to customize the recording duration, for example, to modify the maximum recording duration to 10 seconds, call the SetMaxMessageLength API to set it after initialization.
Flowchart for using the voice message service
Flowchart for using the speech-to-text service
|
| Gets the authentication information |
| Specifies the maximum length of voice message |
Generating the local authentication key
Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment, use the backend deployment key as detailed in Authentication Key. API prototype
GenAuthBuffer(appId: string,roomId: string, openId:string, appKey: string) :string
|
| | AppId from the Tencent Cloud console.
|
| | Enter null or an empty string |
| | User ID, which is the same as OpenId during initialization. |
| | Permission key from the Tencent Cloud console. |
Application authentication
After the authentication information is generated, the authentication is assigned to the SDK.
API prototype
ApplyPTTAuthbuffer(authBuffer: string) :number
|
| | Authentication information |
Sample code
var authBuffer = m_context.GetAuthBuffer(UserConfig.GetAppID(), UserConfig.GetUserID(), null,UserConfig.GetAuthKey());
m_context.ApplyPTTAuthbuffer(authBuffer);
Specifying the maximum duration of voice message
This API is used to specify the maximum duration of a voice message, which can be up to 58 seconds.
API prototype
PttSetMaxMessageLength(msTime: number) :number
|
| | Audio duration in ms. Value range: 1000 < msTime <= 58000 |
Sample code
m_context.PttSetMaxMessageLength(58000);
Streaming Speech Recognition
Voice messaging and speech-to-text APIs
|
PttStartRecordingWithStreamingRecognition | Starts streaming recording |
| Stops recording |
Starting streaming speech recognition
This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. It can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation. To stop recording, call Stop recording. API prototype
PttStartRecordingWithStreamingRecognition(filePath: string, speechLanguage: string, translateLanguage: string) :number
|
| | Path of stored audio file |
| | |
| | |
Sample code
string filePath = "xx/xxx/xxx.silk"
var ret = m_context.StartRecordingWithStreamingRecognition(filePath,"cmn-Hans-CN","cmn-Hans-CN");
if (ret == 0) {
this.currentStatus = "Start streaming recording";
} else {
this.currentStatus = "Failed to start streaming recording";
}
Note:
Translation incurs additional fees. For more information, see Purchase Guide. Callback for streaming speech recognition
After streaming speech recognition is started, you need to listen on callback messages in the OnEvent notification, which is as detailed below:
ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.\nITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING returns the recognized text in real time during the recording, which is equivalent to returning the recognized text while speaking.
The event message will be identified in the callback notification based on the actual needs. The passed parameters include the following four messages.
|
| Return code indicating whether streaming speech recognition is successful |
| Text converted from speech |
| Local path of stored recording file |
| Backend URL address of recording file, which will be retained for 90 days |
Note:
The file_id is empty when the 'ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRecognition_IS_RUNNING' message is listened.
Error codes
|
| Streaming speech-to-text conversion failed, but recording succeeded. | Call the UploadRecordedFile API to upload the recording file and then call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed, but recording and upload succeeded. | The message returned contains a backend URL after successful upload. Call the SpeechToText API to perform speech-to-text conversion. |
| Streaming speech-to-text conversion failed. | During streaming recording, wait for the execution result of the streaming recording API to return. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE:
{
HandleSTREAM2TEXTComplete(data,true);
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_IS_RUNNING:
{
HandleSTREAM2TEXTComplete(data, false);
break;
}
}
});
Voice Message Recording
The recording process is as follows: start recording > stop recording > return recording callback > start the next recording.
Voice messaging and speech-to-text APIs
Starting recording
This API is used to start recording.
API prototype
PttStartRecording(filePath: string) : number;
|
| | Path of stored audio file |
Sample code
string filepath = "xxxx/xxx.silk";
var ret = m_context.PttStartRecording(filepath);
Stopping recording
This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.
API prototype
PttStopRecording() :number;
Sample code
m_context.PttStopRecording();
Callback for recording start
A callback will be executed through a delegate function to pass a message when recording is completed.
To stop recording, call StopRecording. The callback for recording start will be returned after the recording is stopped.
|
| | 0: Recording is completed |
| | Path of stored recording file, which must be accessible and cannot be the fileid |
Error codes
|
| | Check whether the API parameters in the code are correct. |
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| Recording is in progress. | Ensure that the SDK recording feature is used at the right time. |
| Audio data is not captured. | Check whether the mic is working properly. |
| An error occurred while accessing the file during recording. | Ensure the existence of the file and the validity of the file path. |
| The mic is not authorized. | Mic permission is required for using the SDK. To add the permission, see the SDK project configuration document for the corresponding engine or platform. |
| The recording duration is too short. | The recording duration should be in ms and longer than 1,000 ms. |
| No recording operation is started. | Check whether the recording starting API has been called. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_RECORD_COMPLETE:
{
// Process
break;
}
}
});
Pausing recording
This API is used to pause recording. If you want to resume recording, call the PttResumeRecording API.
API prototype
PttPauseRecording() : number
Sample code
number ret = m_context.PttPauseRecording();
Resuming recording
This API is used to resume recording.
API prototype
PttResumeRecording() : number;
Sample code
number ret = m_context.PttResumeRecording();
Canceling recording
This API is used to cancel recording. There is no callback after cancellation.
API prototype
PttCancelRecording() : number
Sample code
m_context.PttCancelRecording();
Voice Message Upload, Download, and Playback
|
| |
| |
| |
| Stops playing back an audio file |
| |
| Gets the audio file duration |
Uploading an audio file
This API is used to upload an audio file.
API prototype
PttUploadRecordedFile(filePath: string) : number
|
| | Path of uploaded audio file, which is a local path. |
Sample code
var ret = m_context.PttUploadRecordedFile(filePath);
Callback for audio file upload completion
After the audio file is uploaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
| | 0: Recording is completed
|
| | Path of stored recording file |
| | |
Error codes
|
| An error occurred while accessing the file during upload. | Ensure the existence of the file and the validity of the file path. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| A network error occurred. | Check whether the device can access the internet. |
| The network failed while getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| The packet returned during the process of getting the upload parameters is empty. | Check whether the authentication is correct and whether the device network can normally access the internet. |
| Failed to decode the packet returned during the process of getting the upload parameters. | Check whether the authentication is correct and whether the device can access the internet. |
| | Check whether the apply API is called or whether the input parameters are empty. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_UPLOAD_COMPLETE:
{
// Process
break;
}
}
});
Downloading the audio file
This API is used to download an audio file.
API prototype
PttDownloadRecordedFile(fileId: string, filePath: string) : number
|
| | |
| | Local path of saved file, which must be accessible and cannot be the fileid |
Sample code
var ret = m_context.PttDownloadRecordedFile(fileID,filePath);
Callback for audio file download completion
After the audio file is downloaded, the event message ITMG_MAIN_EVNET_TYPE_PTT_DOWNLOAD_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path, and file_id.
|
| | |
| | Path of stored recording file |
| | URL of recording file, which will be retained on the server for 90 days. |
Error codes
|
| An error occurred while accessing the file during download. | Check whether the file path is valid. |
| Signature verification failed. | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
| Network storage system exception. | The server failed to get the audio file. Check whether the API parameter fileid is correct, whether the network is normal, and whether the file exists in COS. |
| Server file system error. | Check whether the device can access the internet and whether the file exists on the server. |
| The HTTP network failed during the process of getting the download parameters. | Check whether the device can access the internet. |
| The packet returned during the process of getting the download parameters is empty. | Check whether the device can access the internet. |
| Failed to decode the packet returned during the process of getting the download parameters. | Check whether the device can access the internet. |
| | Check whether the authentication key is correct and whether the voice message and speech-to-text feature is initialized. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
}
});
Playing back audio
This API is used to play back audio.
API prototype
PttPlayRecordedFile(filePath: string, voiceType: ITMG_VOICE_TYPE) : number
Error codes
|
| | Ensure the existence of the file and the validity of the file path. |
Sample code
m_context.PlayRecordedFile(filePath);
Callback for audio playback
After the audio is played back, the event message ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameter includes result and file_path.
|
| | |
| | Path of stored recording file |
Error codes
|
| | Check whether the device is being used, whether the permissions are normal, and whether the initialization is normal. |
| During playback, the client tried to interrupt and play back the next one but failed (which should succeed normally). | Check whether the code logic is correct. |
| | Check whether the API parameters in the code are correct. |
| | An error occurred while initializing the player. This error code is generally caused by failure in decoding, and the error should be located with the aid of logs. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVNET_TYPE_PTT_PLAY_COMPLETE:
{
// Process
break;
}
}
});
Stopping audio playback
This API is used to stop audio playback. There will be a callback for playback completion when the playback stops.
API prototype
PttStopPlayFile() : number
Sample code
m_context.PttStopPlayFile();
Getting audio file size
This API is used to get the size of an audio file.
API prototype
PttGetFileSize(filePath: string) : number
|
| | Path of audio file, which is a local path |
Sample code
m_context.PttGetFileSize(filePath);
Getting audio file duration
This API is used to get the duration of an audio file in milliseconds.
API prototype
PttGetVoiceFileDuration(filePath: string) : number
|
| | Path of audio file, which is a local path |
Sample code
number fileDuration = m_context.PttGetVoiceFileDuration(filePath);
Fast Recording-to-Text Conversion
Translating audio file into text in specified language
This API can specify a language for recognition or translate the information recognized in speech into a specified language and return the translation.
Note:
Translation incurs additional fees. For more information, see Purchase Guide. API prototype
PttSpeechToText(fileID: string, speechLanguage: string, translateLanguage: string) : number
|
| | URL of audio file, which will be retained on the server for 90 days |
| | |
| | |
Sample code
m_context.PttSpeechToText(filePath,"cmn-Hans-CN","cmn-Hans-CN");
Callback for recognition
After the specified audio file is converted to text, the event message ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE will be returned, which will be identified in the OnEvent function.\nThe passed parameters include result, file_path and text (recognized text).
|
| | 0: Recording is completed
|
| | URL of recording file, which will be retained on the server for 90 days |
| | |
Error codes
|
| An internal error occurred. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the device can access the internet. |
| Failed to decode the returned packet. | Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| | Check whether the authentication key is correct and whether the voice messaging and speech-to-text feature is initialized. |
| | Check whether authbuffer is correct. |
| Incorrect speech-to-text conversion parameter. | Check whether the API parameter fileid in the code is empty. |
| Speech-to-text translation returned an error. | Error with the backend of voice messaging and speech-to-text feature. Analyze logs, get the actual error code returned from the backend to the client, and ask backend personnel for assistance. |
| Speech-to-text conversion succeeded, but the text translation service was not activated. | Activate the text translation service in the console. |
| Speech-to-text conversion succeeded, but the language parameter of the text translation service was invalid. | Check the parameter passed in. |
Sample code
m_context.setTMGDelegate(function(eventId, msg){
switch (eventType) {
case ITMG_MAIN_EVENT_TYPE_ENTER_ROOM:
{
// Process
break;
}
...
case ITMG_MAIN_EVNET_TYPE_PTT_SPEECH2TEXT_COMPLETE:
{
// Process
break;
}
});
Voice Message Volume Level APIs
|
| Gets real-time mic volume level |
| Sets recording volume level |
| Gets recording volume level |
| Gets real-time speaker volume level |
| |
| Gets playback volume level |
Getting the real-time mic volume of voice message
This API is used to get the real-time mic volume. A number-type value will be returned. Value range: 0–200.
API prototype
Sample code
m_context.PttGetMicLevel();
Setting the recording volume of voice message
This API is used to set the recording volume of voice message. Value range: 0-200.
API prototype
PttSetMicVolume(vol:number) :number
|
| | Value range: 0–200. Default value: 100. 0 indicates that the audio is mute, while 100 indicates that the volume level remains unchanged. |
Sample code
m_context.PttSetMicVolume(vol);
Getting the recording volume of voice message
This API is used to get the recording volume of voice message. A number-type value will be returned. Value range: 0–200.
API prototype
PttGetMicVolume() : number
Sample code
m_context.PttGetMicVolume();
Getting the real-time speaker volume of voice message
This API is used to get the real-time speaker volume. A number-type value will be returned. Value range: 0-200.
API prototype
PttGetSpeakerLevel() : number;
Sample code
m_context.PttGetSpeakerLevel();
Setting the playback volume of voice message
This API is used to set the playback volume of voice message. Value range: 0-200.
API prototype
PttSetSpeakerVolume(vol: number) : number
Sample code
m_context.PttSetSpeakerVolume(100);
Getting the playback volume of voice message
This API is used to get the playback volume of voice message. A number-type value will be returned. Value range: 0-200.
API prototype
PttGetSpeakerVolume() : number
Sample code
m_context.PttGetSpeakerVolume();
Advanced APIs
Getting version number
This API is used to get the SDK version number for analysis.
API prototype
Sample code
string sdkVersion = m_context.GetSDKVersion();
Setting log printing level
This API is used to set the level of logs to be printed, and needs to be called before the initialization. It is recommended to keep the default level.
API prototype
SetLogLevel(level: number) : number
Parameter description
|
| | Sets the log level. TMG_LOG_LEVEL_NONE indicates not to log. Default value: TMG_LOG_LEVEL_INFO. |
level description:
|
| |
| Prints error logs (default) |
| |
| |
| |
Sample code
m_context.SetLogLevel(TMG_LOG_LEVEL_INFO);
Setting the log printing path
This API is used to set the log printing path. The default path is as follows. It needs to be called before Init.
|
| %appdata%\\GMEGLOBAL\\GME\\ProcessName |
API prototype
SetLogPath(logPath: string)
Sample code
string logDir = ""// Set a path by yourself
m_context.SetLogPath(logDir);
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/a98b45c2b96711ed9e14525400088f3a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/b9c9a7f1b96711ed9e14525400088f3a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/ca7c85c8b96711ed9e14525400088f3a.png\"/)
![\"image\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/e5a9a7f3b96711eda534525400c56988.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/e094c397b37211eeb2a1525400170219.gif\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/f85f96e5b96711eda534525400c56988.png\"/)
![\"image\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/5069068cb37111eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/97b6f432b37111eeae9a525400c26da5.gif\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/5061db5db37111eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/054d8febb37211eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/05198543b37211eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/053d8990b37211ee9939525400461a83.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/ed77e24fd76a11edbe2a525400088f3a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/ac215e26d76b11ed8c12525400c56988.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/b3ed0409d76b11ed8c12525400c56988.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d6acbd3b36e11eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d67dd45b36e11ee9939525400461a83.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d686df5b36e11eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d6ed2c0b36e11eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d6e56c8b36e11ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d6f0491b36e11eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d769a09b36e11ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d700c7cb36e11eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d7e37beb36e11ee9939525400461a83.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d90ccbeb36e11eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3d723e5db36e11ee9939525400461a83.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/89ec8c71d76911edbe2a525400088f3a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/89bb38ead76911edbe2a525400088f3a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/89c48bded76911ed8c12525400c56988.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/89b6421bd76911edbe2a525400088f3a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/b2c4226cb35311ee9939525400461a83.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/b2c38b31b35311eeae9a525400c26da5.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/b2d62f06b35311eeae9a525400c26da5.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/119d2a88b37211ee9fd6525400bb593a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/11a80d2db37211eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/11f72fb3b37211ee9939525400461a83.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/11a6a345b37211ee9fd6525400bb593a.png\"/)
![\"image\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/e2104cf4b34c11eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/29daa5edb34d11eeae9a525400c26da5.gif\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/e20f83b3b34c11ee9fd6525400bb593a.png\"/)
![\"image\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/26ebd42cb34811ee9939525400461a83.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/26eb8c1db34811eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/26ec06f1b34811eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/26f242d3b34811ee9939525400461a83.png\"/)
![\"Image\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/26f72f98b34811ee9fd6525400bb593a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/1cbe39a2b34611ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/1cbe73bcb34611ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/1cbe1bcbb34611eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/1e903f53b34711ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/1e974765b34711eeae9a525400c26da5.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/1e90a8aeb34711eeb2a1525400170219.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/346ac7c2b34311ee9939525400461a83.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/346c02beb34311eeb2a1525400170219.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/346e0b7fb34311eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/d294c022b34411eeae9a525400c26da5.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/d2958c2ab34411eeae9a525400c26da5.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/d2b81568b34411ee9939525400461a83.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/fa8cf27ab34d11eeae9a525400c26da5.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/fa90336bb34d11eeb2a1525400170219.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/17ac6650b34e11ee9939525400461a83.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/17ac2919b34e11ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/17ad5228b34e11ee9fd6525400bb593a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/a616837bb35111eeb2a1525400170219.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/a6197373b35111eeae9a525400c26da5.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/a6479e3bb35111ee9939525400461a83.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/23913b3fb96711eda534525400c56988.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/31a16f32b96711eda534525400c56988.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/3f9c852ab96711ed9e14525400088f3a.png\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/2f896e21b35911ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/2f93cb8ab35911ee9fd6525400bb593a.jpeg\"/)
![\"\"](\"https://cloudcache.intl.tencent-cloud.com/cms/backend-cms/2f9dd21db35911ee9939525400461a83.jpeg\"/)