Overview
This document describes the schemes for migrating a self-built Kafka cluster to a CKafka cluster. You can choose an appropriate scheme according to your own business scenarios.
Scheme Description
Scheme 1. Single-Producer dual-consumer
This scheme is simple, clear, and easy to implement, with no data heap for smooth transition.

Scheme ideas:
- Migrate the topic metadata.
- Keep the original consumer in the self-built Kafka cluster intact.
- Start a new consumer in CKafka and configure the bootstrap-server of the new CKafka cluster for consumption.
- Wait for all consumers to listen on the new CKafka cluster.
- Switch the production of the self-built cluster to the new CKafka cluster (by configuring the bootstrap-server of the new CKafka cluster).
- Deactivate the original consumer in the self-built Kafka cluster after it finishes consuming the remaining data in the cluster.
Pros and cons of the scheme:
- Pros: the overall migration process is simple, clear, and easy to implement, with no data heap for smooth transition.
- Cons: an additional consumer is required.
Scheme 2. Single-Producer single-consumer
This scheme is simple, clear, and easy to implement.

Scheme ideas:
- Migrate the topic metadata.
- Switch the production of the self-built Kafka cluster to the new CKafka cluster (by configuring the bootstrap-server of the new CKafka cluster).
- Wait for the consumer in the self-built cluster to consume the remaining data.
- Switch the old consumer to the new CKafka cluster for consumption (by configuring the bootstrap-server of the new CKafka cluster).
Pros and cons of the scheme:
- Pros: the overall migration process is simple, clear, and easy to implement for smooth transition.
- Cons: after the production is switched to the CKafka cluster, before the old consumer is switched to the CKafka cluster, there will be a certain amount of data heap in the CKafka cluster.
Scheme 3. Migration with MirrorMaker
This scheme migrates the existing data in the self-built Kafka cluster to CKafka.

Scheme ideas:
- Migrate the topic metadata.
- Keep the original consumer in the self-built Kafka cluster intact.
- Start the data sync feature of MirrorMaker.
- Wait for the data sync to complete, modify the consumer configuration, and switch the consumer.
- Wait for the data sync to complete, modify the producer configuration, and switch the producer.
- The migration is completed.
Pros and cons of the scheme:
- Pros: the overall migration process is simple, clear, and easy to implement, where historical data can be synced to the CKafka cluster.
- Cons: the consumer needs to consume data from the beginning after switch to the destination cluster, so consumption idempotency must be ensured.
Was this page helpful?