tencent cloud

Tencent Kubernetes Engine

Release Notes and Announcements
Release Notes
Announcements
Release Notes
Product Introduction
Overview
Strengths
Architecture
Scenarios
Features
Concepts
Native Kubernetes Terms
Common High-Risk Operations
Regions and Availability Zones
Service Regions and Service Providers
Open Source Components
Purchase Guide
Purchase Instructions
Purchase a TKE General Cluster
Purchasing Native Nodes
Purchasing a Super Node
Getting Started
Beginner’s Guide
Quickly Creating a Standard Cluster
Examples
Container Application Deployment Check List
Cluster Configuration
General Cluster Overview
Cluster Management
Network Management
Storage Management
Node Management
GPU Resource Management
Remote Terminals
Application Configuration
Workload Management
Service and Configuration Management
Component and Application Management
Auto Scaling
Container Login Methods
Observability Configuration
Ops Observability
Cost Insights and Optimization
Scheduler Configuration
Scheduling Component Overview
Resource Utilization Optimization Scheduling
Business Priority Assurance Scheduling
QoS Awareness Scheduling
Security and Stability
TKE Security Group Settings
Identity Authentication and Authorization
Application Security
Multi-cluster Management
Planned Upgrade
Backup Center
Cloud Native Service Guide
Cloud Service for etcd
TMP
TKE Serverless Cluster Guide
TKE Registered Cluster Guide
Use Cases
Cluster
Serverless Cluster
Scheduling
Security
Service Deployment
Network
Release
Logs
Monitoring
OPS
Terraform
DevOps
Auto Scaling
Containerization
Microservice
Cost Management
Hybrid Cloud
AI
Troubleshooting
Disk Full
High Workload
Memory Fragmentation
Cluster DNS Troubleshooting
Cluster kube-proxy Troubleshooting
Cluster API Server Inaccessibility Troubleshooting
Service and Ingress Inaccessibility Troubleshooting
Common Service & Ingress Errors and Solutions
Engel Ingres appears in Connechtin Reverside
CLB Ingress Creation Error
Troubleshooting for Pod Network Inaccessibility
Pod Status Exception and Handling
Authorizing Tencent Cloud OPS Team for Troubleshooting
CLB Loopback
API Documentation
History
Introduction
API Category
Making API Requests
Elastic Cluster APIs
Resource Reserved Coupon APIs
Cluster APIs
Third-party Node APIs
Relevant APIs for Addon
Network APIs
Node APIs
Node Pool APIs
TKE Edge Cluster APIs
Cloud Native Monitoring APIs
Scaling group APIs
Super Node APIs
Other APIs
Data Types
Error Codes
TKE API 2022-05-01
FAQs
TKE General Cluster
TKE Serverless Cluster
About OPS
Hidden Danger Handling
About Services
Image Repositories
About Remote Terminals
Event FAQs
Resource Management
Service Agreement
TKE Service Level Agreement
TKE Serverless Service Level Agreement
Contact Us
Glossary

Optimization for High Concurrency Scenarios

PDF
Mode fokus
Ukuran font
Terakhir diperbarui: 2024-08-12 17:48:23

Operation Scenarios

This document introduces how to configure and optimize Nginx Ingress for high concurrency scenarios.

Operation Guide

Increasing CLB Specifications and Bandwidth

High concurrency scenarios require high traffic throughput and forwarding performance of CLB. You can manually create a CLB in the CLB Console, select LCU-supported instance specifications, choose the model as needed, and increase the bandwidth limit (ensure the VPC is consistent with that of the TKE cluster).
After the CLB is created, configure Nginx Ingress to reuse this CLB as the traffic entry. For details, refer to Custom Definition CLB.

Tuning Kernel Parameters and Nginx Configuration

Optimize kernel parameters and Nginx configuration for high concurrency scenarios. values.yaml configuration method:
controller:
extraInitContainers:
- name: sysctl
image: busybox
imagePullPolicy: IfNotPresent
securityContext:
privileged: true
command:
- sh
- -c
- |
sysctl -w net.core.somaxconn=65535 # Increase connection queue to prevent queue overflow.
sysctl -w net.ipv4.ip_local_port_range="1024 65535" # Expand the source port range to prevent port exhaustion.
sysctl -w net.ipv4.tcp_tw_reuse=1 # Enable TIME_WAIT reuse to allow new connections after port exhaustion.
sysctl -w fs.file-max=1048576 # Increase the file handle count to prevent connection overflow from exhausting file handles.
config:
# The number of requests that can be processed by a persistent connection between Nginx and the client is 100 by default. We recommend increasing this number in high-concurrency scenarios, but setting it too high may cause uneven load after Nginx Ingress scale-out.
# Reference: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#keep-alive-requests
keep-alive-requests: "1000"
# The maximum number of idle persistent connections (not the maximum number of connections) between Nginx and the upstream is 320 by default. We recommend increasing this number in high-concurrency scenarios to prevent the frequent establishment of connections from significantly increasing the number of TIME_WAIT connections.
# Reference: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#upstream-keepalive-connections
upstream-keepalive-connections: "2000"
# The maximum number of connections that each worker process can open is 16384 by default.
# Reference: https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/configmap/#max-worker-connections
max-worker-connections: "65536"
Note:

Log Rotation

Nginx Ingress will print logs to the container's standard output by default, which will be managed automatically by the container during running. In high-concurrency scenarios, this may lead to high CPU occupancy.
The solution is to output Nginx Ingress logs to log files and use a sidecar to automatically rotate the log files, preventing the disk space from being filled up with logs.
values.yaml configuration method:
controller:
config:
# Nginx logs are written to log files to avoid high CPU utilization under high concurrency.
access-log-path: /var/log/nginx/nginx_access.log
error-log-path: /var/log/nginx/nginx_error.log
extraVolumes:
- name: log # Log mounting directory of the controller
emptyDir: {}
extraVolumeMounts:
- name: log # Log directory shared by the logrotate and controller
mountPath: /var/log/nginx
extraContainers: # Logrotate sidecar container for log rotation
- name: logrotate
image: imroc/logrotate:latest # https://github.com/imroc/docker-logrotate
imagePullPolicy: IfNotPresent
env:
- name: LOGROTATE_FILE_PATTERN # Pattern of rotated log files, matching the log file path configured in Nginx
value: "/var/log/nginx/nginx_*.log"
- name: LOGROTATE_FILESIZE # Threshold of log file size for rotation
value: "100M"
- name: LOGROTATE_FILENUM # Number of rotations per log file
value: "3"
- name: CRON_EXPR # Crontab expression for periodic logrotate running, which is once every minute
value: "*/1 * * * *"
- name: CROND_LOGLEVEL # Crond log level, ranging from 0 to 8, the smaller the value, the more detailed
value: "8"
volumeMounts:
- name: log
mountPath: /var/log/nginx



Bantuan dan Dukungan

Apakah halaman ini membantu?

masukan