tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Tencent Real-Time Communication
    Documentation
    Download PDF

    Android, iOS, Windows, and macOS

    Last updated: 2023-10-08 15:56:25
    Download PDF
      This document describes how to use the TRTC SDK to implement custom audio capturing and rendering.

      Custom Audio Capturing

      The custom audio capturing feature of the TRTC SDK can be used in two steps: enabling the feature and sending audio frames to the SDK. For detailed directions of specific APIs, see below. We also provide API examples for different platforms:
      Android
      iOS
      Windows

      Enabling custom audio capturing

      To enable the custom audio capturing feature of the TRTC SDK, you need to call the enableCustomAudioCapture API of TRTCCloud. Below is the sample code:
      Android
      iOS&Mac
      Windows 
      TRTCCloud mTRTCCloud = TRTCCloud.shareInstance();
      mTRTCCloud.enableCustomAudioCapture(true);
      
      self.trtcCloud = [TRTCCloud sharedInstance];
      [self.trtcCloud enableCustomAudioCapture:YES];
      
      liteav::ITRTCCloud* trtc_cloud = liteav::ITRTCCloud::getTRTCShareInstance();
      trtc_cloud->enableCustomAudioCapture(true);
      

      Sending custom audio frames

      You can use the sendCustomAudioData API of TRTCCloud to populate the TRTC SDK with your own audio data. Below is the sample code:
      Android
      iOS&Mac
      Windows 
      TRTCCloudDef.TRTCAudioFrame trtcAudioFrame = new TRTCCloudDef.TRTCAudioFrame();
      trtcAudioFrame.data = data;
      trtcAudioFrame.sampleRate = sampleRate;
      trtcAudioFrame.channel = channel;
      trtcAudioFrame.timestamp = timestamp;
      mTRTCCloud.sendCustomAudioData(trtcAudioFrame);
      
      TRTCAudioFrame *audioFrame = [[TRTCAudioFrame alloc] init];
      audioFrame.channels = audioChannels;
      audioFrame.sampleRate = audioSampleRate;
      audioFrame.data = pcmData;
      
      [self.trtcCloud sendCustomAudioData:audioFrame];
      
      liteav::TRTCAudioFrame frame;
      frame.audioFormat = liteav::TRTCAudioFrameFormatPCM;
      frame.length = buffer_size;
      frame.data = array.data();
      frame.sampleRate = 48000;
      frame.channel = 1;
      getTRTCShareInstance()->sendCustomAudioData(&frame);
      
      notice
      Using sendCustomAudioData may cause AEC to fail.

      Getting Raw Audio Data

      The audio module is a highly complex module, and the TRTC SDK needs to strictly control the capturing and playback logic of audio devices. In some cases, to get the audio data of a remote user or audio captured by the local mic, you can use the APIs of TRTCCloud for different platforms. We also provide API examples for those platforms:
      Android:
      iOS
      Windows

      Setting audio callback

      Android
      iOS&Mac
      Windows
      mTRTCCloud.setAudioFrameListener(new TRTCCloudListener.TRTCAudioFrameListener() {
              @Override
              public void onCapturedRawAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {
      
              }
      
              @Override
              public void onLocalProcessedAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {
      
              }
      
              @Override
              public void onRemoteUserAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame, String s) {
      
              }
      
              @Override
              public void onMixedPlayAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {
      
              }
      
              @Override
              public void onMixedAllAudioFrame(TRTCCloudDef.TRTCAudioFrame trtcAudioFrame) {
                  // For more information, see the custom rendering tool class `com.tencent.trtc.mediashare.helper.CustomFrameRender` in `TRTC-API-Example`  
              }
          }); 
      
       [self.trtcCloud setAudioFrameDelegate:self];
       // MARK: - TRTCAudioFrameDelegate
       - (void)onCapturedRawAudioFrame:(TRTCAudioFrame *)frame {
              NSLog(@"onCapturedRawAudioFrame");
      }
      
      - (void)onLocalProcessedAudioFrame:(TRTCAudioFrame *)frame {
              NSLog(@"onLocalProcessedAudioFrame");
      }
      
      - (void)onRemoteUserAudioFrame:(TRTCAudioFrame *)frame userId:(NSString *)userId {
              NSLog(@"onRemoteUserAudioFrame");
      }
      
      - (void)onMixedPlayAudioFrame:(TRTCAudioFrame *)frame {
              NSLog(@"onMixedPlayAudioFrame");
      }
      
      - (void)onMixedAllAudioFrame:(TRTCAudioFrame *)frame {
              NSLog(@"onMixedAllAudioFrame");
      }
      
      // Set custom audio data callback
      liteav::ITRTCCloud* trtc_cloud = liteav::ITRTCCloud::getTRTCShareInstance();
      trtc_cloud->setAudioFrameCallback(callback)
      
      // Callback APIs for custom audio
      
      virtual void onCapturedRawAudioFrame(TRTCAudioFrame* frame) {
      }
      
      virtual void onLocalProcessedAudioFrame(TRTCAudioFrame* frame) {
      }
      
      virtual void onPlayAudioFrame(TRTCAudioFrame* frame, const char* userId) {
      }
      
      virtual void onMixedPlayAudioFrame(TRTCAudioFrame* frame) {
      }
      
      notice
      Do not perform time-consuming operations with any of the above callback functions. We recommend that you copy the data to another thread for processing to avoid AEC failure and choppy audio.
      The data called back by the above callback functions can only be read and copied. Modifying the data may lead to unexpected results.
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy