tencent cloud

文档反馈

自建集群

最后更新时间:2024-01-23 15:44:44

    接入自建集群

    本文介绍接入自建集群的步骤,您可以将自建集群接入容器安全服务进行统一管理,对自建集群开展集群风险检查和管理。

    限制条件

    接入自建集群节点规模小于500节点。

    操作步骤

    1. 登录 容器安全服务控制台,在左侧导航中,单击集群风险管理 > 集群检查
    2. 在集群检查页面,单击接入自建集群
    
    
    3. 在集群信息设置页面,配置相关参数,单击下一步
    
    
    参数说明:
    参数组
    参数
    说明
    可选项
    基础信息设置
    集群名称
    输入自建集群的名称,64字符以内
    -
    集群环境
    选择自建集群的类型
    Kubernetes、Openshift
    集群版本
    选择集群环境的集群版本
    K8s 集群支持1.13以上版本
    网络信息设置
    网络类型
    选择通过公网或通过 VPC 网络接入自建集群
    公网、VPC
    所在地域
    选择自建集群所在的地域,公网类型无地域限制
    -
    VPC ID
    当网络类型使用 VPC 时,选择集群所在网络的 VPC 信息
    -
    API Server地址
    当网络类型使用 VPC 时,选择集群 API Server 后端服务类型
    服务器、负载均衡
    集群检查组件
    安装检查组件
    选择自动或者自行手动安装集群检查的组件
    自动安装检查组件并进行一次集群检查
    不安装检查组件,接入后自行安装组件并下发集群安装
    自动检查
    是否开启集群的自动检查功能
    开启
    关闭
    
    4. 在上传配置文件,单击选择文件,上传本地文件后,单击完成接入即可接入自建集群。
    注意:
    公网方式接入自建集群,如果您的集群有设置访问控制策略,需单击 IP 白名单地址添加页面中的 IP 地址。
    您需要在服务器上生成 K8s 配置文件后,才能上传该配置文件。生成 K8s 配置文件的具体操作,请参见 生成 K8S 配置文件
    上传配置文件,大小需要在1M 以内。
    

    生成 K8s 配置文件

    本文指导您生成容器安全需要的最小化权限 K8s 配置文件。您可参照文档步骤生成配置文件,或者参见 一键脚本

    前提条件

    已在服务器上搭建 K8s 集群。具体操作,请参见 K8s 官方文档
    已安装 Docker 服务。

    操作步骤

    1. 以 root 身份登录 k8s 集群 master 所在服务器。
    2. 输入如下命令,创建命名空间和权限绑定。
    # 1. 创建命名空间: tcss
    # 2. 创建命名空间tcss下的管理角色: tcss-admin
    # 3. 绑定角色tcss-admin和用户tcss
    # 4. 创建秘钥并绑定服务账号: tcss-agent-secret,tcss-agent
    # 5. 创建只读的集群角色: security-clusterrole
    # 6. 绑定集群角色security-clusterrole到服务账号tcss-agent
    
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
    name: tcss
    
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
    namespace: tcss
    name: tcss-admin
    rules:
    - apiGroups: ["extensions", "apps", ""]
    resources: ["*"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
    name: tcss-admin-rb
    namespace: tcss
    subjects:
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    roleRef:
    kind: Role
    name: tcss-admin
    apiGroup: rbac.authorization.k8s.io
    
    
    ---
    apiVersion: v1
    kind: Secret
    metadata:
    name: tcss-agent-secret
    namespace: tcss
    annotations:
    kubernetes.io/service-account.name: tcss-agent
    type: kubernetes.io/service-account-token
    
    
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
    name: tcss-agent
    namespace: tcss
    secrets:
    - name: tcss-agent-secret
    namespace: tcss
    
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
    name: security-clusterrole
    rules:
    - apiGroups: ["", "v1"]
    resources: ["namespaces", "pods", "nodes"]
    verbs: ["get", "list"]
    - apiGroups: ["apps"]
    resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
    verbs: ["get", "list"]
    - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"]
    verbs: ["get", "list"]
    - apiGroups: ["rbac.authorization.k8s.io"]
    resources: ["clusterroles", "clusterrolebindings"]
    verbs: ["get"]
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
    name: security-clusterrolebinding
    roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: security-clusterrole
    subjects:
    - kind: ServiceAccount
    name: tcss-agent
    namespace: tcss
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    3. 进入 k8s 配置目录(/etc/kubernetes/),输入如下命令,创建证书。
    # 创建User私钥 tcss.key。
    openssl genrsa -out tcss.key 2048
    
    # 创建证书签署请求 tcss.csr
    openssl req -new -key tcss.key -out tcss.csr -subj "/O=K8s/CN=tcss"
    
    # 签署证书 生成 tcss.crt
    openssl x509 -req -in tcss.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tcss.crt -days 365
    4. 输入如下命令,创建集群配置文件。
    # 创建并设置集群配置, 其中需要 API Server 地址必须为公网可访问地址
    kubectl config set-cluster tcss --server=https://xx.xx.xx.xx:60002 --certificate-authority=/etc/kubernetes/ca.crt --embed-certs=true --kubeconfig=/root/tcss.conf
    
    # 创建并设置用户配置
    kubectl config set-credentials tcss --client-certificate=tcss.crt --client-key=tcss.key --embed-certs=true --kubeconfig=/root/tcss.conf
    
    # 设置context配置
    kubectl config set-context tcss@tcss --cluster=tcss --user=tcss --kubeconfig=/root/tcss.conf
    
    # 切换context配置
    kubectl config use-context tcss@tcss --kubeconfig=/root/tcss.conf
    5. 验证集群配置文件并上传配置。
    KUBECONFIG=/root/tcss.conf kubectl -n tcss get pod
    说明:
    执行上述命令,如果能显示 pod 或者显示当前命名空间下没有相关资源,则表示该集群配置是可用的,上传该文件 /root/tcss.conf 即可。

    一键脚本

    在 master 节点中,您可基于以下一键脚本代码一键快速生成集群配置文件:
    说明:
    环境需要提前安装 openssl。
    #!/bin/bash
    
    set -e;
    
    # API_SERVER 需要设置为公网可访问的地址和端口
    # API_SERVER=https://xx.xx.xx.xx:xxxx
    
    # 以下路径,用户根据集群实际情况设定
    KUBECONFIG_TARGET=/root/tcss.conf
    CA_FILE=/etc/kubernetes/ca.crt
    CAKEY_FILE=/etc/kubernetes/ca.key
    TCSS_TMPDIR=/tmp/tcss
    # 如果是OpenShift环境,可以更换为 oc
    KUBECTL_CMD=kubectl
    
    
    if [ ! $API_SERVER ]; then
    echo "API_SERVER does not set.";
    exit 1;
    fi
    if ! which kubectl ; then
    echo "kubectl does not exist.";
    exit 1;
    fi
    if [ ! -f "$CA_FILE" ]; then
    echo "$CA_FILE does not exist.";
    exit 1;
    fi
    if [ ! -f "$CAKEY_FILE" ]; then
    echo "$CAKEY_FILE does not exist.";
    exit 1;
    fi
    if [ ! -d $TCSS_TMPDIR ]; then
    mkdir -p $TCSS_TMPDIR;
    fi
    
    cat <<EOF > $TCSS_TMPDIR/tcss_res.yaml
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
    name: tcss
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
    namespace: tcss
    name: tcss-admin
    rules:
    - apiGroups: ["extensions", "apps", ""]
    resources: ["*"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
    name: tcss-admin-rb
    namespace: tcss
    subjects:
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    roleRef:
    kind: Role
    name: tcss-admin
    apiGroup: rbac.authorization.k8s.io
    
    ---
    apiVersion: v1
    kind: Secret
    metadata:
    name: tcss-agent-secret
    namespace: tcss
    annotations:
    kubernetes.io/service-account.name: tcss-agent
    type: kubernetes.io/service-account-token
    
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
    name: tcss-agent
    namespace: tcss
    secrets:
    - name: tcss-agent-secret
    namespace: tcss
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
    name: security-clusterrole
    rules:
    - apiGroups: ["", "v1"]
    resources: ["namespaces", "pods", "nodes"]
    verbs: ["get", "list"]
    - apiGroups: ["apps"]
    resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
    verbs: ["get", "list"]
    - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"]
    verbs: ["get", "list"]
    - apiGroups: ["rbac.authorization.k8s.io"]
    resources: ["clusterroles", "clusterrolebindings"]
    verbs: ["get"]
    
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
    name: security-clusterrolebinding
    roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: security-clusterrole
    subjects:
    - kind: ServiceAccount
    name: tcss-agent
    namespace: tcss
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    EOF
    
    # echo "generate tcss resource file ($TCSS_TMPDIR/tcss_res.yaml) success."
    
    $KUBECTL_CMD apply -f $TCSS_TMPDIR/tcss_res.yaml;
    
    # 创建User私钥 tcss.key。
    openssl genrsa -out $TCSS_TMPDIR/tcss.key 2048
    # 创建证书签署请求 tcss.csr
    openssl req -new -key $TCSS_TMPDIR/tcss.key -out $TCSS_TMPDIR/tcss.csr -subj "/O=K8s/CN=tcss"
    # 签署证书 生成 tcss.crt
    openssl x509 -req -in $TCSS_TMPDIR/tcss.csr -CA $CA_FILE -CAkey $CAKEY_FILE -CAcreateserial -out $TCSS_TMPDIR/tcss.crt -days 365
    
    # 创建并设置集群配置
    $KUBECTL_CMD config set-cluster tcss --server=$API_SERVER --certificate-authority=$CA_FILE --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
    # 创建并设置用户配置
    $KUBECTL_CMD config set-credentials tcss --client-certificate=$TCSS_TMPDIR/tcss.crt --client-key=$TCSS_TMPDIR/tcss.key --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
    # 设置context配置
    $KUBECTL_CMD config set-context tcss@tcss --cluster=tcss --user=tcss --kubeconfig=$KUBECONFIG_TARGET
    # 切换context配置
    $KUBECTL_CMD config use-context tcss@tcss --kubeconfig=$KUBECONFIG_TARGET
    
    echo "generate KUBECONFIG file success. $KUBECONFIG_TARGET"

    生成 Openshift 配置文件

    本文指导您生成容器安全需要的最小化权限 OpenShift 配置文件。您可参照文档步骤生成配置文件,或者参见 一键脚本

    前提条件

    1. 已在服务器上搭建 K8s 集群。具体操作,请参见 K8s 官方文档
    2. 已安装 Docker 服务。
    3. 暂仅支持 OpenShift3.0及以上版本接入,低于该版本可能存在不确定性问题。

    操作步骤

    说明:
    整体接入思路和 Kubernetes 类似,只涉及相关路径和命令行工具区别,如果集群 master 节点上已经安装的 kubectl 工具,则可以完全同 Kubernetes 集群接入方式进行接入。
    1. 以 root 身份登录 OpenShift 集群 master 所在服务器。
    2. 输入如下命令,创建命名空间和权限绑定。
    # 1. 创建命名空间: tcss
    # 2. 创建命名空间tcss下的管理角色: tcss-admin
    # 3. 绑定角色tcss-admin和用户tcss
    # 4. 创建秘钥并绑定服务账号: tcss-agent-secret,tcss-agent
    # 5. 创建只读的集群角色: security-clusterrole
    # 6. 绑定集群角色security-clusterrole到服务账号tcss-agent
    
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
    name: tcss
    
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
    namespace: tcss
    name: tcss-admin
    rules:
    - apiGroups: ["extensions", "apps", ""]
    resources: ["*"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
    name: tcss-admin-rb
    namespace: tcss
    subjects:
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    roleRef:
    kind: Role
    name: tcss-admin
    apiGroup: rbac.authorization.k8s.io
    
    
    ---
    apiVersion: v1
    kind: Secret
    metadata:
    name: tcss-agent-secret
    namespace: tcss
    annotations:
    kubernetes.io/service-account.name: tcss-agent
    type: kubernetes.io/service-account-token
    
    
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
    name: tcss-agent
    namespace: tcss
    secrets:
    - name: tcss-agent-secret
    namespace: tcss
    
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
    name: security-clusterrole
    rules:
    - apiGroups: ["", "v1"]
    resources: ["namespaces", "pods", "nodes"]
    verbs: ["get", "list"]
    - apiGroups: ["apps"]
    resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
    verbs: ["get", "list"]
    - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"]
    verbs: ["get", "list"]
    - apiGroups: ["rbac.authorization.k8s.io"]
    resources: ["clusterroles", "clusterrolebindings"]
    verbs: ["get"]
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
    name: security-clusterrolebinding
    roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: security-clusterrole
    subjects:
    - kind: ServiceAccount
    name: tcss-agent
    namespace: tcss
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    3. 进入配置目录(/etc/origin/master/),输入如下命令,创建证书。
    # 创建User私钥 tcss.key。
    openssl genrsa -out tcss.key 2048
    
    # 创建证书签署请求 tcss.csr
    openssl req -new -key tcss.key -out tcss.csr -subj "/O=K8s/CN=tcss"
    
    # 签署证书 生成 tcss.crt
    openssl x509 -req -in tcss.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out tcss.crt -days 365
    4. 输入如下命令,创建集群配置文件。
    # 创建并设置集群配置, 其中需要主要 server 地址必须为公网可访问地址
    KUBECONFIG=/root/tcss.conf oc config set-cluster tcss --server=https://xx.xx.xx.xx:60002 --certificate-authority=/etc/origin/master/ca.crt --embed-certs=true --kubeconfig=/root/tcss.conf
    
    # 创建并设置用户配置
    KUBECONFIG=/root/tcss.conf oc config set-credentials tcss --client-certificate=tcss.crt --client-key=tcss.key --embed-certs=true --kubeconfig=/root/tcss.conf
    
    # 设置context配置
    KUBECONFIG=/root/tcss.conf oc config set-context tcss@tcss --cluster=tcss --user=tcss --kubeconfig=/root/tcss.conf
    
    # 切换context配置
    KUBECONFIG=/root/tcss.conf oc config use-context tcss@tcss --kubeconfig=/root/tcss.conf
    5. 输入如下命令,验证集群配置文件并上传配置。
    KUBECONFIG=/root/tcss.conf oc -n tcss get pod
    说明:
    执行上述命令,如果能显示pod或者显示当前命名空间下没有相关资源则表示该集群配置是可用的,上传该文件 /root/tcss.conf 即可。

    一键脚本

    在 master 节点中,您可基于以下一键脚本代码一键快速生成集群配置文件:
    说明:
    环境需要提前安装 openssl。
    #!/bin/bash
    
    set -e;
    
    # API_SERVER 需要设置为公网可访问的地址和端口
    # API_SERVER=https://xx.xx.xx.xx:xxxx
    
    # 以下路径,用户根据集群实际情况设定
    KUBECONFIG_TARGET=/root/tcss.conf
    CA_FILE=/etc/kubernetes/ca.crt
    CAKEY_FILE=/etc/kubernetes/ca.key
    TCSS_TMPDIR=/tmp/tcss
    KUBECTL_CMD=oc
    
    
    if [ ! $API_SERVER ]; then
    echo "API_SERVER does not set.";
    exit 1;
    fi
    if ! which $KUBECTL_CMD ; then
    echo "$KUBECTL_CMD does not exist.";
    exit 1;
    fi
    if [ ! -f "$CA_FILE" ]; then
    echo "$CA_FILE does not exist.";
    exit 1;
    fi
    if [ ! -f "$CAKEY_FILE" ]; then
    echo "$CAKEY_FILE does not exist.";
    exit 1;
    fi
    if [ ! -d $TCSS_TMPDIR ]; then
    mkdir -p $TCSS_TMPDIR;
    fi
    
    cat <<EOF > $TCSS_TMPDIR/tcss_res.yaml
    ---
    apiVersion: v1
    kind: Namespace
    metadata:
    name: tcss
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: Role
    metadata:
    namespace: tcss
    name: tcss-admin
    rules:
    - apiGroups: ["extensions", "apps", ""]
    resources: ["*"]
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: RoleBinding
    metadata:
    name: tcss-admin-rb
    namespace: tcss
    subjects:
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    roleRef:
    kind: Role
    name: tcss-admin
    apiGroup: rbac.authorization.k8s.io
    
    ---
    apiVersion: v1
    kind: Secret
    metadata:
    name: tcss-agent-secret
    namespace: tcss
    annotations:
    kubernetes.io/service-account.name: tcss-agent
    type: kubernetes.io/service-account-token
    
    ---
    apiVersion: v1
    kind: ServiceAccount
    metadata:
    name: tcss-agent
    namespace: tcss
    secrets:
    - name: tcss-agent-secret
    namespace: tcss
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRole
    metadata:
    name: security-clusterrole
    rules:
    - apiGroups: ["", "v1"]
    resources: ["namespaces", "pods", "nodes"]
    verbs: ["get", "list"]
    - apiGroups: ["apps"]
    resources: ["replicasets", "daemonsets", "deployments", "statefulsets"]
    verbs: ["get", "list"]
    - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"]
    verbs: ["get", "list"]
    - apiGroups: ["rbac.authorization.k8s.io"]
    resources: ["clusterroles", "clusterrolebindings"]
    verbs: ["get"]
    
    
    ---
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
    name: security-clusterrolebinding
    roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: security-clusterrole
    subjects:
    - kind: ServiceAccount
    name: tcss-agent
    namespace: tcss
    - kind: User
    name: tcss
    apiGroup: rbac.authorization.k8s.io
    EOF
    
    # echo "generate tcss resource file ($TCSS_TMPDIR/tcss_res.yaml) success."
    
    $KUBECTL_CMD apply -f $TCSS_TMPDIR/tcss_res.yaml;
    $KUBECTL_CMD adm policy add-scc-to-user privileged -n tcss -z tcss-agent;
    $KUBECTL_CMD adm policy add-scc-to-user hostaccess -n tcss -z tcss-agent;
    $KUBECTL_CMD adm policy add-scc-to-user privileged tcss;
    $KUBECTL_CMD adm policy add-scc-to-user hostaccess tcss;
    oc adm policy add-cluster-role-to-user cluster-reader tcss;
    
    # 创建User私钥 tcss.key。
    openssl genrsa -out $TCSS_TMPDIR/tcss.key 2048
    # 创建证书签署请求 tcss.csr
    openssl req -new -key $TCSS_TMPDIR/tcss.key -out $TCSS_TMPDIR/tcss.csr -subj "/O=K8s/CN=tcss"
    # 签署证书 生成 tcss.crt
    openssl x509 -req -in $TCSS_TMPDIR/tcss.csr -CA $CA_FILE -CAkey $CAKEY_FILE -CAcreateserial -out $TCSS_TMPDIR/tcss.crt -days 365
    
    # 创建并设置集群配置
    KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config set-cluster tcss --server=$API_SERVER --certificate-authority=$CA_FILE --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
    # 创建并设置用户配置
    KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config set-credentials tcss --client-certificate=$TCSS_TMPDIR/tcss.crt --client-key=$TCSS_TMPDIR/tcss.key --embed-certs=true --kubeconfig=$KUBECONFIG_TARGET
    # 设置context配置
    KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config set-context tcss@tcss --cluster=tcss --user=tcss --kubeconfig=$KUBECONFIG_TARGET
    # 切换context配置
    KUBECONFIG=$KUBECONFIG_TARGET $KUBECTL_CMD config use-context tcss@tcss --kubeconfig=$KUBECONFIG_TARGET
    
    echo "generate KUBECONFIG file success. $KUBECONFIG_TARGET"
    
    联系我们

    联系我们,为您的业务提供专属服务。

    技术支持

    如果你想寻求进一步的帮助,通过工单与我们进行联络。我们提供7x24的工单服务。

    7x24 电话支持