This document provides a detailed description that makes it easy for Native project developers to debug and integrate the APIs for Game Multimedia Engine (GME).
This document only provides the main APIs to help you get started with GME to debug and integrate the APIs.
GME provides two services: Voice chat service and voice messaging and speech-to-text service, both of which rely on key APIs such as Init and Poll.
Note on Init APIIf you need to use voice chat and voice messaging services at the same time, you only need to call
Init
API once.
Initializing GMEAPI: Init
Calling Poll periodically to trigger event callbacksAPI: Poll
On the SDK download guide page, download the appropriate client SDKDownLoad.
import com.tencent.TMG.ITMGContext;
import com.tencent.av.sig.AuthBuffer;
import com.tencent.bugly.crashreport.CrashReport;
To use the voice feature, get the ITMGContext
object first.
public static ITMGContext GetInstance(Context context)
//MainActivity.java
import com.tencent.TMG.ITMGContext;
ITMGContext tmgContext = ITMGContext.GetInstance(this);
The API class uses the Delegate
method to send callback notifications to the application. Register the callback function to the SDK for receiving callback messages before room entry.
Register the callback function to the SDK for receiving callback messages before room entry.
//ITMGContext
public abstract int SetTMGDelegate(ITMGDelegate delegate);
//MainActivity.java
tmgContext.SetTMGDelegate(TMGCallbackDispatcher.getInstance());
Override this callback function in the constructor to process the parameters of the callback.
//MainActivity.java
tmgContext.SetTMGDelegate(TMGCallbackDispatcher.getInstance());
//RealTimeVoiceActivity.java
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
{
// Processing callbacks
}
}
// Refer to TMGCallbackDispatcher.java, TMGCallbackHelper.java, and TMGDispatcherBase.java
Parameter | Type | Description |
---|---|---|
type | ITMGContext.ITMG_MAIN_EVENT_TYPE | Event type in the callback response |
data | Intent message type | Callback message, i.e., event data |
You need to initialize the SDK through the Init
API before you can use the real-time voice, voice message, and speech-to-text services. The Init
API must be called in the same thread as other APIs. We recommend you call all APIs in the main thread.
public abstract int Init(String sdkAppId, String openId);
Parameter | Type | Description |
---|---|---|
sdkAppId | string | AppID provided in the GME console, which can be obtained as instructed in Activating Services. |
openID | string | openID can only be in Int64 type, which is passed in after being converted to a string. You can customize its rules, and it must be unique in the application. To pass in openID as a string, submit a ticket for application. |
//MainActivity.java
int nRet = tmgContext.Init(appId, openId);
if (nRet == AV_OK )
{
GMEAuthBufferHelper.getInstance().setGEMParams(appId, key, openId);
// Step 4/11: Poll to trigger callback
//https://www.tencentcloud.com/document/product/607/40860
EnginePollHelper.createEnginePollHelper();
showToast("Init success");
}else if (nRet == AV_ERR_HAS_IN_THE_STATE) // SDK has been initialized. This operation is successful.
{
showToast("Init success");
}else
{
showToast("Init error errorCode:" + nRet);
}
Event callbacks can be triggered by periodically calling the Poll
API in update
. The Poll
API is GME's message pump and should be called periodically for GME to trigger event callbacks; otherwise, the entire SDK service will run abnormally. For more information, see the EnginePollHelper
file in SDK Download Guide.
//MainActivity.java
[EnginePollHelper createEnginePollHelper];
//EnginePollHelper.java
private Handler mhandler = new Handler();
private Runnable mRunnable = new Runnable() {
@Override
public void run() {
if (s_pollEnabled) {
if (ITMGContext.GetInstance(null) != null)
ITMGContext.GetInstance(null).Poll();
}
mhandler.postDelayed(mRunnable, 33);
}
};
// For the code of calling Poll periodically, see EnginePollHelper.java.
Generate AuthBuffer
for encryption and authentication of relevant features. For release in the production environment, please use the backend deployment key as detailed in Authentication Key.
AuthBuffer public native byte[] genAuthBuffer(int sdkAppId, String roomId, String openId, String key)
Parameter | Type | Description |
---|---|---|
appId | int | AppId from the Tencent Cloud console. |
roomId | string | Room ID, which can contain up to 127 characters (For voice message, enter "null".) |
openId | string | User ID, which is the same as openId during initialization. |
key | string | Permission key from the Tencent Cloud console. |
//GMEAuthBufferHelper.java
import com.tencent.av.sig.AuthBuffer;// Header file
public byte[] createAuthBuffer(String roomId)
{
byte[] authBuffer;
// Generate AuthBuffer for encryption and authentication of relevant features. For release in the production environment,
// please use the backend deployment key as detailed in https://www.tencentcloud.com/document/product/607/12218
if (TextUtils.isEmpty(roomId))
{
authBuffer = AuthBuffer.getInstance().genAuthBuffer(Integer.parseInt(mAppId), "0", mOpenId, mKey);
}else
{
authBuffer = AuthBuffer.getInstance().genAuthBuffer(Integer.parseInt(mAppId), roomId, mOpenId, mKey);
}
return authBuffer;
}
This API is used to enter a room with the generated authentication information. The mic and speaker are not turned on by default after room entry. The returned value of AV_OK
indicates successful API call but not successful room entry.
public abstract int EnterRoom(String roomID, int roomType, byte[] authBuffer);
Parameter | Type | Description |
---|---|---|
roomId | String | Room ID, which can contain up to 127 characters |
roomType | int | Use FLUENCY sound quality to enter the room |
authBuffer | byte[] | Authentication code |
//RealTimeVoiceActivity.java
byte[] authBuffer = GMEAuthBufferHelper.getInstance().createAuthBuffer(roomId);
ITMGContext.GetInstance(this).EnterRoom(roomId, roomType, authBuffer);
After the user enters the room, the message ITMG_MAIN_EVENT_TYPE_ENTER_ROOM
will be sent and identified in the OnEvent
function for callback and processing. A successful callback means that the room entry is successful, and the billing starts.
//RealTimeVoiceActivity.java
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (type == ITMG_MAIN_EVENT_TYPE_ENTER_ROOM)
{
// Step 6/11 : Perform the enter room event
int nErrCode = TMGCallbackHelper.ParseIntentParams2(data).nErrCode;
String strMsg = TMGCallbackHelper.ParseIntentParams2(data).strErrMsg;
if (nErrCode == AV_OK)
{
appendLog2MonitorView("EnterRomm success");
}else
{
appendLog2MonitorView(String.format(Locale.getDefault(), "EnterRomm errCode:%d errMsg:%s", nErrCode, strMsg));
}
}
}
Error Code Value | Cause and Suggested Solution |
---|---|
7006 | Authentication failed. Possible causes:
|
7007 | Already in another room. |
1001 | The user was already in the process of entering a room but repeated this operation. It is recommended not to call the room entering API until the room entry callback is returned. |
1003 | The user was already in the room and called the room entering API again. |
1101 | Make sure that the SDK is initialized, `openId` complies with the rules, the APIs are called in the same thread, and the `Poll` API is called normally. |
This API is used to turn on or off the mic. Mic and speaker are not enabled by default after room entry.
//RealTimeVoiceActivity.java
ITMGContext.GetInstance(this).GetAudioCtrl().EnableMic(true);
This API is used to turn on/off the speaker.
//RealTimeVoiceActivity.java
ITMGContext.GetInstance(this).GetAudioCtrl().EnableSpeaker(true);
This API is called to exit the current room. It needs to wait for and process the callback for exit.
//RealTimeVoiceActivity.java
ITMGContext.GetInstance(this).ExitRoom();
After the user exits a room, a callback will be returned with the message being ITMG_MAIN_EVENT_TYPE_EXIT_ROOM
. The sample code is shown below:
//RealTimeVoiceActivity.java
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVENT_TYPE_EXIT_ROOM == type)
{
// Receive the event of successful room exit
}
}
Call authentication initialization after initializing the SDK. For more information on how to get the authBuffer
, please see genAuthBuffer
(the voice chat authentication information API).
public abstract int ApplyPTTAuthbuffer(byte[] authBuffer);
Parameter | Type | Description |
---|---|---|
authBuffer | String | Authentication |
//VoiceMessageRecognitionActivity.java
byte[] authBuffer = GMEAuthBufferHelper.getInstance().createAuthBuffer("");
ITMGContext.GetInstance(this).GetPTT().ApplyPTTAuthbuffer(authBuffer);
This API is used to start streaming speech recognition. Text obtained from speech-to-text conversion will be returned in real time in its callback. To stop recording, call StopRecording
. The callback will be returned after the recording is stopped.
public abstract int StartRecordingWithStreamingRecognition (String filePath);
public abstract int StopRecording();
Parameter | Type | Description |
---|---|---|
filePath | String | Path of stored audio file |
//VoiceMessageRecognitionActivity.java
ITMGContext.GetInstance(this).GetPTT().StartRecordingWithStreamingRecognition(recordfilePath);
After streaming speech recognition is started, you need to listen for callback messages in the callback function onEvent
. The event message is ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE
, namely returns text after the recording is stopped and the recognition is completed, which is equivalent to returning the recognized text after a paragraph of speech.
The event message will be identified in the OnEvent function
based on the actual needs. The passed parameters include the following four messages.
Message Name | Description |
---|---|
result | A return code for judging whether the streaming speech recognition is successful. |
text | Text converted from speech |
file_path | Local path of stored recording file |
file_id | Backend URL address of recording file, which will be retained for 90 days |
//VoiceMessageRecognitionActivity.java
import static com.tencent.TMG.ITMGContext.ITMG_MAIN_EVENT_TYPE.ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE;
public void OnEvent(ITMGContext.ITMG_MAIN_EVENT_TYPE type, Intent data) {
if (type == ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE)
{
// Step 1.3/3 handle ITMG_MAIN_EVNET_TYPE_PTT_STREAMINGRECOGNITION_COMPLETE event
mIsRecording = false;
if (nErrCode ==0)
{
String recordfilePath = data.getStringExtra("file_path");
mRecFilePathView.setText(recordfilePath);
String recordFileUrl = data.getStringExtra("file_id");
mRecFileUrlView.setText(recordFileUrl);
}
else
{
appendLog2MonitorView("Record and recognition fail errCode:" + nErrCode);
}
}
}
Error Code | Description | Suggested Solution |
---|---|---|
32775 | Streaming speech-to-text conversion failed, but recording succeeded. | Call the `UploadRecordedFile` API to upload the recording file and then call the `SpeechToText` API to perform speech-to-text conversion. |
32777 | Streaming speech-to-text converting failed, but recording and upload succeeded | The message returned contains a backend URL after successful upload. Call the `SpeechToText` API to perform speech-to-text conversion. |
32786 | Streaming speech-to-text conversion failed. | During streaming recording, wait for the execution result of the streaming recording API to return. |
This API is used to stop recording. It is async, and a callback for recording completion will be returned after recording stops. A recording file will be available only after recording succeeds.
public abstract int StopRecording();
//VoiceMessageRecognitionActivity.java
ITMGContext.GetInstance(this).GetPTT().StopRecording();
Was this page helpful?