Creating a Topic
Last updated:2025-12-24 14:59:00
Scenarios
Topic is a core concept in TDMQ for Apache Pulsar. Typically, topics are used for centralized classification and management of various types of messages produced by the system. For example, messages related to transactions can be stored in a topic named "trade" for other consumers to subscribe to.
In actual application scenarios, a topic often represents a business aggregation, and developers determine how to design different topics based on their system design and data architecture design.
This document describes how to use topics for message classification and management in TDMQ for Apache Pulsar.
Constraints and Limitations
Regardless of whether a topic is created manually or automatically, it occupies the topic quota in a cluster. When the total number of topics reaches the upper limit, no topic can be created.
If your cluster was created before January 18, 2025, automatic topic creation is not supported. To enable this feature, contact us to upgrade the cluster version.
Only pro clusters support the automatic topic creation feature.
Prerequisites
Operation Steps
1. Log in to the TDMQ for Apache Pulsar console, and click Topic in the left sidebar.
2. On the Topic page, select the region, cluster, and namespace, and click Create to open the Create Topic dialog box.
3. In the Create Topic dialog box, specify the following information:
Parameter | Required | Description |
Region | Yes | It is automatically specified based on the region where the selected cluster is located. You do not need to specify it. |
Namespace | Yes | It is automatically specified based on the selected namespace. You do not need to specify it. |
Topic Name | Yes | It is a string of no more than 128 characters, containing digits, letters, and special characters "-_=:.". |
Topic Type | Yes | The options include Persistent and Non-persistent, and the default value is Persistent. Persistent: Messages in persistent topics are persisted to the disk with multiple replicas to ensure that the messages are not lost. Persistent topics apply to high-reliability scenarios, such as finance and transactions. Non-persistent: Messages in non-persistent topics are not persisted to the disk and are delivered to online subscribers directly, and they are deleted immediately after delivery. If no subscribers are online, the messages are deleted directly and not retained on the server. Non-persistent topics apply to scenarios with low data reliability requirements and stream processing. (Messages in non-persistent topics only support immediate sending/receiving of general messages and do not support features, such as message query, message tracing, delayed messages, message filtering, and message rollback). Note: For non-persistent topics, you should specify the complete topic name prefixed with non-persistent:// during message sending or receiving. |
Pa | No | It can be enabled and disabled, and it is enabled by default. When it is enabled, the default number of partitions is 2. When the partition switch is disabled, a topic is not divided into different partitions, and the number of partitions does not need to be configured. When the partition switch is enabled, a topic is divided into different partitions, and the number of partitions needs to be configured. The value range is 1 to 32. In TDMQ for Apache Pulsar, messages are sequential in a single partition. That is, if a topic with 1 partition is created, messages are globally sequential. A single-partition topic has poorer performance than a multi-partition topic. To balance performance and sequence, see Subscription Modes and use the Key-Shared mode for consumption. This ensures local sequence. Assign the same key to messages requiring sequential processing, ensuring that they are routed to the same partition. |
Isolate Abnormal Consumer | No | It can be enabled and disabled, and it is disabled by default. When it is enabled, the default acknowledgment timeout period is 1 minute. TDMQ for Apache Pulsar allows the server to proactively isolate abnormal consumers, with an optional acknowledgment timeout period. The acknowledgment timeout period can be set from 1 minute to 24 hours. For scenarios and use cases about abnormal consumer isolation, see Abnormal Consumer Isolation. Enable this feature only after you clearly understand the applicable scenarios of this feature to avoid unnecessary impact on your production system. |
Description | No | Enter the topic description, with a string of no more than 128 characters. |
4. Click Save. You can see the created topic in the topic list.
Parameter | Description |
Topic Name | Topic name, in the format of pulsar-****/namespace/topicName. |
Monitoring | Click  |
Type | The options include Persistent and Non-persistent. |
Creator | User-created or system-created. |
Number of Partitions | Number of partitions in a topic. |
Client | Producer: Displays the value of Number of existing producers in a cluster/Maximum number of producers supported in a cluster. Click to go to the production details page. For details, see Viewing Producer Connection Relationships. Consumer: Displays the value of Number of existing consumers in a cluster/Maximum number of consumers supported in a cluster. Click to go to the consumption details page. For details, see Viewing and Managing Subscription Relationships. Note: When the orange color that represents warning is displayed, the score has reached 80%. When the red color that represents error is displayed, the score has reached 90%. Promptly disconnect any unused client connections. |
Creation Time | Creation time of a topic. |
Description | Topic description. |
Automatic topic creation indicates that when a client produces messages to or consumes messages in a topic, the system automatically checks whether the topic exists. If the topic does not exist and its name complies with naming rules, the system automatically creates the topic.
Enabling Automatic Topic Creation
1. Log in to the TDMQ for Apache Pulsar console.
2. In the left sidebar, choose Cluster, select the target region, and click the ID of the target cluster to go to the Basic Information page.
3. On the Basic Information page, click Edit in the upper-right corner under the Automatic Topic Creation module.
4. In the pop-up window, enable Automatic Topic Creation and configure the default number of partitions.
In TDMQ for Apache Pulsar, messages are sequential in a single partition. That is, if a topic with 1 partition is created, messages are globally sequential.
Increasing the number of partitions can improve the production and consumption performance of a single topic. However, the message sequentiality cannot be guaranteed.
5. Click Confirm to complete the enabling.
Client Usage Methods
1. When the automatic topic creation feature is used, ensure that topic names comply with the following naming conventions:
Topic names should be a string of up to 128 characters.
Topic names should only contain valid characters (digits, letters, and special characters "-_=:.").
Topic names cannot contain the string "-partition-".
2. When the automatic topic creation feature is used, you can only create persistent partition topics.
3. Avoid creating persistent topics and non-persistent topics with identical names.
If you create a non-persistent topic (non-persistent://tenant/namespace/topic1) in the console and then use automatic topic creation to create a persistent topic (persistent://tenant/namespace/topic1), the persistent topic1 will not be displayed in the console and will affect monitoring data reporting.
If this issue occurs, you can delete the non-persistent topic1 in the console. After the page is refreshed, data is synchronized, and the persistent topic1 is displayed.
4. When the retry and dead letter queues are used and the automatic topic creation feature is enabled, note the following:
If users do not configure the
retryLetterTopic and deadLetterTopic parameters in deadLetterPolicy in the consumer construction logic, the system automatically creates a retry queue topic named subscriptionName-RETRY and a dead letter queue topic named subscriptionName-DLQ for message sending/receiving. These topics are different from the retry and dead letter queue topics created by enabling the automatic topic creation feature in the console.In addition, if existing users have not configured a complete
deadLetterPolicy and are using clients of earlier versions, restarting the consumer will cause messages to be sent to and consumed from the retry queue (named subscriptionName-RETRY) and dead letter queue (named subscriptionName-DLQ). The originally used retry queue (named topicName-subscriptionName-RETRY) and dead letter queue (named topicName-subscriptionName-DLQ) will become invalid, causing message loss.To avoid the preceding issues, it is recommended that you configure the retry topic and dead letter topic in
deadLetterPolicy when a consumer is created.Consumer<byte[]> consumer = myClient.newConsumer().topic("persistent://wzjtest-tenant/test-ns/topic1").subscriptionName("sub").enableRetry(true).subscriptionType(SubscriptionType.Shared).deadLetterPolicy(DeadLetterPolicy.builder().maxRedeliverCount(1)//Complete configuration.retryLetterTopic("topic1-sub-RETRY") //Naming convention: topicName-subscriptionName-RETRY.deadLetterTopic("topic1-sub-DLQ") //Naming convention: topicName-subscriptionName-DLQ.build()).subscribe();
Java clients of 3.0.6, 3.3.1, 4.0.0, or later versions can avoid this issue by restricting automatic topic creation in the retry queue (named subscriptionName-RETRY) and dead letter queue (named subscriptionName-DLQ). However, it is still recommended that users explicitly configure the complete retry topic and dead letter topic parameters in deadLetterPolicy. This issue cannot be avoided for Go clients of existing versions.
5. The number of topics and the number of partitions are restricted under different cluster specifications.
If the automatic topic creation feature is used, it is necessary to control an appropriate number of topics.
If the number of topics exceeds the maximum topic quantity under cluster specifications, you will be unable to successfully create topics and will receive an error "Topic Not Found."
Common Issues
If you encounter the "Topic Not Found." error when using the automatic topic creation feature, troubleshoot the issue based on the following possible causes:
The topic is non-persistent.
The topic name contains the string "-partition-".
The topic name exceeds the maximum length limit of 128 characters.
The number of created topics has reached the maximum quantity under the cluster specifications.
The topic name contains unsupported characters (characters excluding digits, letters, and special characters "-_=:.").
The topic name does not comply with the three-segment naming standard (persistent://tenantName/namespacesName/topicName).
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No
Feedback