tencent cloud


Technical Principles

Last updated: 2024-01-24 17:20:44

    System Architecture

    The system architecture of the CKafka Connector is as shown below:
    Primarily divided into four layers:

    Data Source

    The data source refers to the location of the customer's data source. The data source can be in the cloud, self-built IDC, cross cloud, or hybrid cloud. The data can be business data, log data, or data in the DB, etc.

    Access Layer

    The CKafka connector provides an access layer that adapts to various protocols, such as HTTP Rest, Kafka Protocol, Change Data Capture, etc. The access layer is deployed in a distributed manner, which features capabilities such as Auto Scaling and automatic retrying, ensuring the stability of data access.

    Storage Layer

    The storage layer of the CKafka connector is the Message Queue (MQ) on Tencent Cloud. By default, it supports Kafka, but it can also support other MQs such as Pulsar, RocketMQ. The MQ storage layer primarily serves to level peak loads, distribute data, and cache.

    Data Distribution Layer

    The CKafka Connector offers a data cleansing (ETL) engine that, based on the configuration, carries out data cleansing and provides the Connector with the dump capability. It can consume data from the data storage layer and guide the data into various different downstream storage engines.
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support