tencent cloud


Multiple-Master Affinity Scheduling

Last updated: 2021-11-18 12:47:30


    In Fluid, you can use nodeselector to specify master node deployment. For example, you can choose to deploy a master to a Kubernetes server with higher performance.

    The feature is introduced as follows.


    Before running the sample code provided in this document, install Fluid by referring to Installation and check that all the components used by Fluid are running properly.

    $ kubectl get pod -n fluid-system
    goosefsruntime-controller-5b64fdbbb-84pc6   1/1     Running   0          8h
    csi-nodeplugin-fluid-fwgjh                  2/2     Running   0          8h
    csi-nodeplugin-fluid-ll8bq                  2/2     Running   0          8h
    dataset-controller-5b7848dbbb-n44dj         1/1     Running   0          8h

    Normally, you shall see a pod named dataset-controller, a pod named goosefsruntime-controller, and multiple pods named csi-nodeplugin. The number of csi-nodeplugin pods depends on the number of nodes in your Kubernetes cluster.

    Setting Up an Environment

    $ mkdir <any-path>/co-locality
    $ cd <any-path>/co-locality


    Check all nodes in your cluster

    $ kubectl get nodes
    NAME                       STATUS   ROLES    AGE     VERSION   Ready    <none>   7d14h  v1.18.4-tke.13   Ready    <none>   7d14h  v1.18.4-tke.13

    Label one of the nodes

    $ kubectl label nodes hbase-cache=true

    Check all nodes again

    $ kubectl get node -L hbase-cache
    NAME                       STATUS   ROLES    AGE     VERSION            HBASE-CACHE   Ready    <none>   7d14h  v1.18.4-tke.13   true   Ready    <none>   7d14h  v1.18.4-tke.13

    Check the Dataset resource object to be created

    apiVersion: data.fluid.io/v1alpha1
    kind: Dataset
     name: hbase
       - mountPoint: https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/stable/
         name: hbase

    To facilitate testing, mountPoint is set to Web UFS in this example. If you want to mount COS, see Mounting COS (COSN) to GooseFS.

    Create the Dataset resource object

    $ kubectl create -f dataset.yaml
    dataset.data.fluid.io/hbase created

    Check the GooseFSRuntime resource object to be created

    apiVersion: data.fluid.io/v1alpha1
    kind: GooseFSRuntime
     name: hbase
     replicas: 1
         - mediumtype: MEM
           path: /dev/shm
           quota: 2G
           high: "0.8"
           low: "0.7"
         hbase-cache: "true"

    The above snippet of the configuration file contains a lot of GooseFS related configuration, and Fluid will start a GooseFS instance based on the configuration. Among the configuration, the spec.replicas attribute is set to 1, indicating that Fluid will start a GooseFS instance containing 1 GooseFS master and 1 GooseFS worker.

    Create the GooseFSRuntime resource object and check its status

    $ kubectl create -f runtime.yaml
    goosefsruntime.data.fluid.io/hbase created
    $ kubectl get pod -o wide
    NAME                 READY   STATUS    RESTARTS   AGE    IP              NODE                       NOMINATED NODE   READINESS GATES
    hbase-fuse-42csf     1/1     Running   0          104s   <none>           <none>
    hbase-master-0       2/2     Running   0          3m3s   <none>           <none>
    hbase-worker-l62m4   2/2     Running   0          104s   <none>           <none>

    As shown above, the master is started successfully and runs on the node with the hbase-cache=true label.

    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support