This document describes how to use the single-producer dual-consumer scheme to migrate data from a self-built Kafka cluster to a CKafka cluster.
If your requirement for message ordering is not high, you can migrate the data while it is consumed by multiple consumers in parallel.
The single-producer dual-consumer scheme is simple, clear, and easy to implement, with no data heap for smooth transition; however, it requires adding a new consumer.
The migration steps are as follows:
--bootstrap-serverby copying the information in the Network column in the Access Mode section on the Instance Details page in the console.
./kafka-console-consumer.sh --bootstrap-server xxx.xxx.xxx.xxx:9092 --from-beginning --new-consumer --topic topicName --consumer.config ../config/consumer.properties
broker-listto the accessed network of the CKafka instance and
topicNameto the topic name in the CKafka instance:
./kafka-console-producer.sh --broker-list xxx.xxx.xxx.xxx:9092 --topic topicName
The above commands are test commands. In actual business operations, just modify the broker address configured for the corresponding application and then restart the application.