tencent cloud

Overview

DataHub supports accessing different types of data generated by various data sources for unified management and distribution to downstream offline/online processing systems, forming a clear data flow channel.

This document takes COS data as an example to describe how to create an async data pull task and modify the task configuration in the CKafka console.

Directions

Creating data access task

Prerequisites

• You have created a CKafka instance and a topic.
• You have created a bucket and an object

Directions:

1. Log in to the CKafka console.

2. Click Data Access on the left sidebar, select the region, and click Create Task.

3. In the pop-up window, select Asynchronously pulled data > COS for Data Source Type.
   

4. Click Next and enter the task details.
   

   • Task Name: It can only contain letters, digits, underscores, or symbols ("-" and ".").
   • Target CKafka Instance: Select a CKafka instance.
   • Target Topic: Select the target CKafka topic for data delivery.
   • Source Bucket: Select the source data bucket.
   • Source Object: Select the source data object. The current topic supports a max message size of 12 MB. If the size of a single data row exceeds this limit, data will not be written to the topic.
   • Role Authorization: You need to grant permissions to a third-party role to access COS.

5. Click Submit.

Changing data source and data target

1. Log in to the CKafka console.
2. Click Data Access on the left sidebar and click the ID of the target task to enter its basic information page.
3. Click Change Data Target in the top-right corner of the Data Target module to modify the data target information.

Viewing monitoring data

1. Log in to the CKafka console.
2. Click Data Access on the left sidebar and click the ID of the target task to enter its basic information page.
3. Select the Monitoring tab to view the monitoring data of the target topic.

Pausing task

On the Data Access page, click Pause in the Operation column of the target task to pause the task.

Note：

If you find that the data access task affects the normal use of CKafka, you can pause it.

Resuming task

On the Data Access page, click Resume in the Operation column of the target task to resume the paused task.

Note：

A paused task can be resumed to continue dumping data.

Deleting task

On the Data Access page, click Delete in the Operation column of the target task and click OK in the pop-up window to delete the task.

Note：

• Once the task is deleted, data access will be stopped and the task record will be deleted, but the previously dumped data and CKafka instance involved will not be affected.
• A task cannot be recovered once deleted. Proceed with caution.