In a metadata migration task, you can sync the metadata of a self-built RocketMQ cluster to TDMQ for RocketMQ. After the sync, you need to modify the access information of the producer and consumer clusters in order to migrate them from the self-built cluster to TDMQ for RocketMQ for message sending and receiving.
Currently, the message migration service migrates only message production and consumption linkages but not message data in the old RocketMQ cluster. It applies only to exclusive clusters as the destination and will be supported for virtual clusters after the beta test ends.
This document describes the double-read double-write and batch release schemes of the message migration service. During migration, the producer and consumer clusters can produce or consume messages in the old RocketMQ and new TDMQ for RocketMQ clusters in parallel. Data will not be heaped because of the migration, so the business can transition smoothly.
The detailed directions are as follows:
- If you don't follow the above steps strictly, for example, if you switch producers first and then consumers, message loss may occur.
- Before switching the remaining consumers, make sure that all messages in the old RocketMQ cluster have been consumed; otherwise, some messages may not be consumed. You can view the number of heaped messages in the old cluster to check whether consumption has completed.
Message sequence cannot be guaranteed due to cluster switch.
Message repetition occurs only in extreme cases. For example, if a consumer consumes a message but doesn't send an acknowledgment to the server (the old RocketMQ cluster), the message will be put into the retry queue, causing repeated consumption. Implementing the idempotency logic can avoid this issue.
During the read switch, partitions are reallocated with rebalancing between the queue and consumer client. This may cause a short consumption delay, but you don't need to handle it because it won’t persist after the switch.
Contact our sales team or business advisors to help your business.
Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.
Was this page helpful?