tencent cloud


Pulsar Multi-AZ Deployment

Last updated: 2024-06-28 11:29:49

    Pulsar Multi-AZ Deployment

    Pulsar supports multi-AZ deployment. When you purchase a Pulsar cluster in a region with three or more AZs, you can select up to three AZs for multi-AZ instance deployment. The partition replicas of this instance will be forcibly distributed to the nodes in each AZ. This deployment mode allows your instance to normally provide services even if it is unavailable in a single AZ.

    How multi-AZ deployment works

    Pulsar implements cross-AZ disaster recovery based on rack awareness. Although the service nodes of different components are deployed in different racks in different AZs, they are essentially in a single Pulsar cluster.
    As a type of ensemble placement policy, rack awareness is the algorithm used by BookKeeper client to select ensembles based on the network topology attribute.

    How to select bookie

    When a bookie cluster is deployed for the first time, each bookie will register a temporary zk-node to the current ZooKeeper, and the BookKeeper client will discover the bookie list via ZooKeeper. When a bookie changes or fails, the ensemble placement policy will be notified of this through the onClusterChanged (Set, Set) API and reconstruct a new network topology. Subsequent operations, such as newEnsemble (used to generate a group of bookies), will be performed based on the new network topology.

    Network topology

    Network topology shows the bookie node information in a cluster in a tree-like hierarchical structure. A bookie cluster may consist of many data centers in different regions. A data center contains bookie nodes distributed on different racks. In the tree-like structure, the leaf nodes show the bookie information.
    Example 1: There are three bookies: bk1, bk2, and bk3 in region A. Their network locations are /region-a/rack-1/bk1, /region-a/rack-1/bk2, and /region-a/rack-2/bk3, respectively, as shown in the network topology below:
    / \\
    rack-1 rack-2
    / \\ \\
    bk1 bk2 bk3
    Example 2: There are four bookies: bk1, bk2, bk3, and bk4 in region A and region B. Their network locations are /region-a/rack-1/bk1, /region-a/rack-1/bk2, /region-b/rack-2/bk3, and /region-b/rack-2/bk4, respectively, as shown in the network topology below:
    / \\
    region-a region-b
    | |
    rack-1 rack-2
    / \\ / \\
    bk1 bk2 bk3 bk4
    The network locations are resolved with the DNSToSwitchMapping method from domain names or IP addresses. They must be in the format of "/region/rack", where the first slash (/) represents root, "region" represents data center, and "rack" represents rack information.
    The DNSToSwitchMapping method implemented in Pulsar is called BookieRackAffinityMapping. It uses ZooKeeper to store rack information and only supports the rack awareness capability currently.

    Use Limit

    Currently, only pro clusters support multi-AZ deployment. For more information, see Product Selection.
    This feature is currently in beta testing and is free of charge.
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support