tencent cloud

Elastic MapReduce

  • Release Notes and Announcements
  • Product Introduction
  • Purchase Guide
    • EMR on CVM Billing Instructions
    • EMR on TKE Billing Instructions
    • EMR Serverless HBase Billing Instructions
    • EMR Serverless TCBase Billing Overview
  • Getting Started
  • EMR on CVM Operation Guide
    • Planning Cluster
    • Administrative rights
    • Configuring Cluster
    • Managing Cluster
    • Managing Service
    • Monitoring and Alarms
    • TCInsight
  • EMR on TKE Operation Guide
  • EMR Serverless HBase Operation Guide
  • EMR Serverless TCBase Operation Guide
  • EMR Development Guide
    • Hadoop Development Guide
    • Spark Development Guide
    • Hbase Development Guide
    • Phoenix on Hbase Development Guide
    • Hive Development Guide
    • Presto Development Guide
    • Sqoop Development Guide
    • Hue Development Guide
    • Oozie Development Guide
    • Flume Development Guide
    • Kerberos Development Guide
    • Knox Development Guide
    • Alluxio Development Guide
    • Kylin Development Guide
    • Livy Development Guide
    • Kyuubi Development Guide
    • Zeppelin Development Guide
    • Hudi Development Guide
    • Superset Development Guide
    • Impala Development Guide
    • Druid Development Guide
    • TensorFlow Development Guide
    • Kudu Development Guide
    • Ranger Development Guide
    • Kafka Development Guide
    • StarRocks Development Guide
    • Flink Development Guide
    • JupyterLab Development Guide
    • MLflow Development Guide
  • Practical Tutorial
    • Practice of EMR on CVM Ops
    • Data Migration
    • Practical Tutorial on Custom Scaling
  • API Documentation
    • History
    • Introduction
    • API Category
    • Making API Requests
    • Cluster Resource Management APIs
    • Cluster Services APIs
    • User Management APIs
    • Information Query APIs
    • Scaling APIs
    • Configuration APIs
    • Other APIs
    • Cluster Lifecycle APIs
    • Serverless HBase APIs
    • YARN Resource Scheduling APIs
    • Data Types
    • Error Codes
  • FAQs
    • EMR on CVM
  • Service Level Agreement
  • Contact Us

Kafka Usage

Download
포커스 모드
폰트 크기
마지막 업데이트 시간: 2025-01-03 15:02:25

Generating Data

By Java code

@Component
@Slf4j
public class KafkaProducer {

@Autowired
private KafkaTemplate<String, Object> kafkaTemplate;

// Custom topics.
public static final String TOPIC_TEST = "topic.test";

//
public static final String TOPIC_GROUP1 = "topic.group1";

//
public static final String TOPIC_GROUP2 = "topic.group2";

public void send(Object obj) {
String obj2String = JSONObject.toJSONString(obj);
log.info("the message to send: {}", obj2String);
// Send a message.
ListenableFuture<SendResult<String, Object>> future = kafkaTemplate.send(TOPIC_TEST, obj);
future.addCallback(new ListenableFutureCallback<SendResult<String, Object>>() {
@Override
public void onFailure(Throwable throwable) {
// Returned result for failed sending
log.info(TOPIC_TEST + " - the producer failed to send the message:" + throwable.getMessage());
}

@Override
public void onSuccess(SendResult<String, Object> stringObjectSendResult) {
// Returned result for successful sending
log.info(TOPIC_TEST + " - the producer sent the message successfully:" + stringObjectSendResult.toString());
}
});
}
}

By command

bin/kafka-console-producer.sh --broker-list node86:9092 --topic t_cdr

Consuming Data

By Java code

@Component
@Slf4j
public class KafkaConsumer {

@KafkaListener(topics = KafkaProducer.TOPIC_TEST, groupId = KafkaProducer.TOPIC_GROUP1)
public void topic_test(ConsumerRecord<?, ?> record, Acknowledgment ack, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {

Optional message = Optional.ofNullable(record.value());
if (message.isPresent()) {
Object msg = message.get();
log.info("topic_test consumed: Topic:" + topic + ",Message:" + msg);
ack.acknowledge();
}
}

@KafkaListener(topics = KafkaProducer.TOPIC_TEST, groupId = KafkaProducer.TOPIC_GROUP2)
public void topic_test1(ConsumerRecord<?, ?> record, Acknowledgment ack, @Header(KafkaHeaders.RECEIVED_TOPIC) String topic) {

Optional message = Optional.ofNullable(record.value());
if (message.isPresent()) {
Object msg = message.get();
log.info("topic_test1 consumed: Topic:" + topic + ",Message:" + msg);
ack.acknowledge();
}
}
}

By command

bin/kafka-console-consumer.sh --zookeeper node01:2181 --topic t_cdr --from-beginning

Adding a topic (by command)

bin/kafka-topics.sh --zookeeper node01:2181 --create --topic t_cdr --partitions 30 --replication-factor 2
For more information, see Kafka Documentation.

도움말 및 지원

문제 해결에 도움이 되었나요?

피드백