In edge computing scenarios, it is common to manage multiple edge sites within the same cluster, with each edge site containing one or more computing nodes.
It is also desirable to run a set of interconnected services within each site, with the services within each site providing a complete set of features for users.
Due to network constraints, interconnected services may not want or be able to access each other across sites.
Considering these three characteristics of edge computing, TKE-Edge has specifically designed a custom resource logic called ServiceGroup to address application distribution and service governance issues encountered in multi-regional scenarios.
Scenarios
ServiceGroup allows for conveniently deploying a set of services within different data centers or regions that belong to the same cluster. It ensures that requests between services can be completed within the same data center or region, avoiding cross-regional access.
Native Kubernetes cannot directly control the specific node locations where Deployment Pods are created. This must be done indirectly by orchestrating node affinity planning. When the number of edge sites and the number of services to be deployed are too high, the management and deployment aspects can become extremely complex or even only theoretically feasible. Furthermore, in order to limit service interactions within a certain range, the business side needs to create dedicated Services for each Deployment, resulting in a huge and error-prone workload that may lead to online business anomalies.
ServiceGroup is designed specifically for this scenario. Users only need to use the two TKE-Edge self-developed Kubernetes resources provided by ServiceGroup, DeploymentGrid and ServiceGrid, to conveniently deploy services to these node groups and control service traffic. This approach also ensures the number of services and disaster recovery in each region.
Architecture
Basic Concept
ServiceGroup needs to be used in conjunction with NodeUnit and NodeGroup. The concepts are detailed as follows:
NodeUnit
NodeUnit typically consists of one or more computing resource instances located within the same edge site. It is essential to ensure that nodes within a NodeUnit have an internal network connection.
For more detail information, please refer to: NodeUnit
NodeGroup
Multiple NodeUnits: NodeGroup includes one or more NodeUnits.
Deploying services on every NodeUnit: It ensures that the services in the ServiceGroup are deployed on every NodeUnit within the group.
Automatic deployment to new NodeUnits: When a NodeUnit is added to the cluster, the services within the ServiceGroup are automatically deployed to the newly added NodeUnit.
For more detail information, please refer to: NodeGroup
ServiceGroup
ServiceGroup is not a definition of a concrete resource, but rather a collection of Kubernetes custom resources. It is an abstract resource, and multiple ServiceGroups can be created within a cluster.
A ServiceGroup contains one or more business services and is applicable to the following scenarios:
Business services need to be packaged for deployment.
Business services need to run in each NodeUnit and ensure a specific number of pods.
Business services need to control the interactions between services within the same NodeUnit, and cannot forward traffic to other NodeUnits.
The resource types involved in ServiceGroup include the following three categories:
DeploymentGrid
StatefulSetGrid
ServiceGrid
The format of DeploymentGrid is similar to Deployment. The <deployment-template> field corresponds to the original template field of the deployment. What is special about DeploymentGrid is the gridUniqKey field, which specifies the key value of the label for node grouping:
apiVersion: superedge.io/v1
kind: DeploymentGrid
metadata:
name:
namespace:
spec:
gridUniqKey: <NodeLabel Key>
<deployment-template>
The format of StatefulSetGrid is similar to StatefulSet. The <statefulset-template> field corresponds to the original template field of the statefulset. What is special about StatefulSetGrid is the gridUniqKey field, which specifies the key value of the label for node grouping:
apiVersion: superedge.io/v1
kind: StatefulSetGrid
metadata:
name:
namespace:
spec:
gridUniqKey: <NodeLabel Key>
<statefulset-template>
The format of ServiceGrid is similar to Service. The <service-template> field corresponds to the original template field of the service. What is special about ServiceGrid is the gridUniqKey field, which specifies the key value of the label for node grouping:
apiVersion: superedge.io/v1
kind: ServiceGrid
metadata:
name:
namespace:
spec:
gridUniqKey: <NodeLabel Key>
<service-template>
Operation Steps
Taking the deployment of Nginx service at the edge as an example, if we want to deploy a complete set of Nginx services separately within multiple node pools, we need to perform the following operations:
Create NodeUnit and NodeGroup
Let's take an edge cluster as an example and add the nodes within the cluster to the edge node pool（NodeUnit） and node pool classification（NodeGroup）. This cluster consists of 4 edge nodes located in 2 regions, Beijing and Guangzhou. The node names are bj-1, bj-2, gz-1, gz-2, as shown in the picture below:
Create 2 NodeUnit: beijing and guangzhou, and add corresponding nodes to the NodeUnit:
Create a NodeGroup (edge node pool classification) named location, and divide the two NodeUnit, beijing and guangzhou, into this classification, as shown in the following image:
After performing the above operations, each node will be labeled accordingly. The labels for node gz-2 are shown in the image below:
Description：
The key of the label is the name of the NodeGroup, and the value is the name of the NodeUnit. Nodes with the same value belong to the same NodeUnit.
If there are multiple NodeGroups within the same cluster, create different NodeGroup names as unique identifiers. When deploying ServiceGroup-related resources, they will be bound to the designated NodeGroup for deployment through the unique identifier of the NodeGroup name.
Stateless ServiceGroup
Create DeploymentGrid
1. Select "ServiceGroup" > "DeploymentGrid"
2. Click "Create", create a DeploymentGrid called "nginx"
NodeGroup: Here, choose the NodeGroup group where you want to deploy the Nginx service in bulk. By selecting location, it means that the corresponding Deployment will be deployed under the two NodeUnits, beijing and guangzhou.
Other parameters: The remaining parameters are identical to the TKE application deployment method, so no further details are provided here.
3. Click "Create DeploymentGrid" and wait for finish.
4. Click on the "nginx" link to enter the detail page, where you can view the specifics of the created Deployment, as shown in the image below:
Introduction：
The platform created a standard Kubernetes Deployment under each NodeUnit contained in the NodeGroup, named as DeploymentGrid-NodeUnit. According to the example, the names here are nginx-beijing and nginx-guangzhou respectively.
Create ServiceGrid
1. Select "ServiceGroup" > "ServiceGrid", click "Create", as follow
NodeGroup: Choose the required NodeGroup, which should be the same NodeGroup as the one selected for DeploymentGrid previously.
Access Settings: Standard Kubernetes Service configuration information. Choose the required port; in this case, the nginx service is set to port 80 by default.
Workload Binding: Choose the Pods required by the Service using a Selector. You can manually enter and add them, or you can select 'Reference Workload' to add the previously deployed DeploymentGrid, in this case > nginx.
2. Click "Create ServiceGrid". After successful creation, the event details page is displayed.
3. You can view the specific created Service information under Service > Service, as shown in the following image:
4. By using the nginx-svc Service, the following objectives can be achieved:
When accessing this Service from a Pod in the Beijing region, only the Pods within the Beijing region will be accessed by the backend.
When accessing this Service from a Pod in the Guangzhou region, only the Pods within the Guangzhou region will be accessed by the backend.
Access to this Service in each region will be restricted within the local NodeUnit scope.
Yes
No
Was this page helpful?