Sarama is a Kafka client written in Go with a high message throughput.
After you actively increase the number of CKafka partitions due to a performance bottleneck, Sarama may be unable to perceive the partition rebalance, causing a failure to produce/consume messages in the new partitions.
After CKafka rebalances partitions for various reasons, it will take about 10 minutes for Sarama to perceive the partition changes and then pull the metadata of the current topic to parse the latest partition data. As the pull is time-consuming, it may be regarded as a failure sometimes.
In addition, the CKafka team has found occasional failures of Sarama to pull the latest partition information, causing message retention and random exception reporting.
This problem has been repeatedly reported and fixed in the Sarama community. It occurs less frequently on the latest version but still exists.
If you are sensitive to rebalance and use the Go technology stack, we recommend you migrate to Confluent-Kafka-Go, a client maintained by Confluent™.
Go Client | Strengths | Limitations |
---|---|---|
Sarama |
|
It is moderately stable. It may experience unknown errors after the number of partitions is increased and all partitions are rebalanced. |
Confluent-Kafka-go (recommended) |
|
|
kafka-go |
|
It underperforms the above two clients and may be unable to sustain a high concurrency. |
Was this page helpful?