ip-masq-agent Overview
Download
Focus Mode
Font Size
Introduction
ip-masq-agent is a network component provided by Tencent Cloud TKE. It is deployed as a DaemonSet to every node (except super nodes) in TKE general clusters and configures iptables rules to implement IP Masquerade for outbound packets from Pods.
IP Masquerading
IP Masquerade is a form of Source Network Address Translation (SNAT). SNAT is the process where, when an internal address accesses external resources, the initiating private IP address is translated into a designated IP address. This allows multiple private addresses to be consolidated into a single address for outbound access.
In a TKE cluster, IP Masquerade applies to all packets sent from Pods. ip-masq-agent configures iptables rules on each node. After a packet from a Pod undergoes IP Masquerade, its source IP address is translated from the Pod IP address to the IP address of the node where the Pod resides.
As shown in the following figure, if we assume that the Pod's outbound destination IP address is 8.8.8.8, its complete outbound path after undergoing IP Masquerade is as follows:
Kubernetes Objects Deployed in a Cluster
Kubernetes Object Name | Type | Default Resource Usage | Associated Namespaces |
ip-masq-agent-config | ConfigMap | - | kube-system |
ip-masq-agent | DaemonSet | - | kube-system |
Use Cases
In TKE general clusters that use the Global Router network mode, when a Pod accesses a network segment outside its own VPC (such as the external Internet), its source address must be translated to the node's IP address. The Pod then uses the network of that node IP for outbound access. Because Pods in the Global Router network mode cannot be bound to EIPs or use NAT Gateways, Pods in this network mode must use IP Masquerade for outbound access.
In a TKE cluster, other scenarios that require using the node IP address for access include authentication and access control.
Usage
ip-masq-agent Configuration Items
The configuration items of ip-masq-agent are stored as a ConfigMap in the kube-system namespace of the cluster. The configuration is periodically hot-reloaded into the component. The interval for hot reloading depends on the ResyncInterval specified in the configuration.
Its configuration values must be written in YAML or JSON format. A simple example is as follows:
apiVersion: v1data:config: |NonMasqueradeCIDRs:- 192.168.0.0/16NonMasqueradeSrcCIDRs:- 10.0.0.0/12MasqLinkLocal: trueMasqLinkLocalIPv6: falseResyncInterval: 1m0skind: ConfigMapmetadata:name: ip-masq-agent-confignamespace: kube-system
NonMasqueradeCIDRs: A list of strings. You can specify network segment CIDRs. IP Masquerade will not be performed when Pods access these destination network segments.
NonMasqueradeSrcCIDRs (supported in version v2.6.2 and later): A list of strings. You can specify network segment CIDRs. Packets sent from Pods within these source network segments in the cluster will not undergo IP Masquerade.
MasqLinkLocal: A Boolean value (true/false) that indicates whether IP Masquerade is performed when Pods in the cluster access the link-local address 169.254.0.0/16.
MasqLinkLocalIPv6: A Boolean value (true/false) that indicates whether IP Masquerade is performed when Pods in the cluster access the link-local address fe80::/10 using the IPv6 protocol.
ResyncInterval: The interval for hot-reloading the configuration. The default value is "1m0s", which means the configuration is loaded and the corresponding IP Masquerade rules are delivered every 60 seconds.
Configuring Pods to Access a CIDR Block (such as 10.0.0.0/12) Without IP Masquerading
1. In the cluster, modify ip-masq-agent-config via the Kubernetes API.
kubectl -nkube-system edit cm ip-masq-agent-config
2. Add 10.0.0.0/12 to NonMasqueradeCIDRs:
3. Enter "
:wq" to save and exit.4. Wait for about one minute (depending on the configured ResyncInterval) for the changes to take effect. You can check the component logs to confirm:
Configuring Pods to Access All CIDR Blocks Without IP address Masquerading
In a cluster using the VPC-CNI network mode, you can configure all Pod outbound traffic to bypass IP address Masquerade. To do this, simply add 0.0.0.0/0 to the NonMasqueradeCIDRs configuration. For detailed steps, refer to the section above.
Best Practices
IP address Masquerade utilizes the system's iptables capability and extends the network path, which incurs some performance overhead. Therefore, it is not recommended to use IP address Masquerade extensively in non-essential scenarios. The following sections provide some best practice recommendations for common network mode scenarios in TKE.
Clusters in Global Router Network Mode
Because the network segment in the Global Router network mode is a virtual segment of the VPC and lacks full VPC network capabilities (such as a NAT Gateway), it is recommended to configure only private network segments in the NonMasqueradeCIDRs for clusters in this mode:
NonMasqueradeCIDRs:- 10.0.0.0/8- 172.16.0.0/12- 192.168.0.0/16
With this configuration, Pods in the Global Router mode do not perform IP Masquerade when accessing private network segments (typically the internal segments of this VPC or other directly reachable segments), and they use the Pod IP address as the source IP. However, when non-private network segments are accessed, IP Masquerade is performed, and the node IP address is used as the source IP.
Clusters in VPC-CNI Network Mode
Because Pods in the VPC-CNI network mode can utilize the full network capabilities of the VPC, it is recommended to minimize the use of IP Masquerade in this mode. When creating a new VPC-CNI mode cluster, you can choose not to install this component. If it is installed, you can also configure the NonMasqueradeCIDRs to 0.0.0.0/0, which means IP Masquerade is disabled for all network segments.
Synchronizing Configuration When a VPC-CNI Container Subnet Is Added
Because the new container subnet for VPC-CNI can reside in the VPC secondary CIDR blocks, and these secondary CIDR blocks might not be included in the NonMasqueradeCIDRs of ip-masq-agent when the cluster is created, it is recommended in most scenarios where IP address Masquerade is not required to add a new VPC-CNI container subnet (Operation path: TKE Console -> choose the cluster ID to go to the specific cluster console -> Node and Network Information -> Add Subnet) and synchronously add the VPC secondary CIDR blocks to the NonMasqueradeCIDRs. If 0.0.0.0/0 is already configured, you can ignore this step.
Currently, TKE supports automatically synchronizing this configuration when a container subnet is added. It is recommended to select this option in most scenarios where the IP address Masquerade feature is not required (if it is not selected, the original configuration is retained):
Clusters Using Both Global Router and VPC-CNI Modes
In a mixed mode, Pods in the Global Router network mode require IP Masquerade when accessing non-private network segments, while Pods in the VPC-CNI mode can access any network segment without IP Masquerade. Therefore, it is recommended to configure private network segments for the NonMasqueradeCIDRs, and configure the VPC CIDR block (excluding the Global Router network segment) for the NonMasqueradeSrcCIDRs. For example, if you set the VPC CIDR blocks to 172.16.0.0/16 and 10.16.0.0/16:
NonMasqueradeCIDRs:- 10.0.0.0/8- 172.16.0.0/12- 192.168.0.0/16NonMasqueradeSrcCIDRs:- 172.16.0.0/16- 10.16.0.0/16
After the configuration is performed according to the method above, Pods in the Global Router mode do not perform IP Masquerade when accessing private network segments, and they use the Pod IP address as the source IP address. However, when accessing non-private network segments, IP Masquerade is performed, and the node IP address is used as the source IP address. Pods in the VPC-CNI mode (with source addresses from the VPC CIDR block) do not perform IP Masquerade when accessing any network segment.
Help and Support
Was this page helpful?
You can also Contact sales or Submit a Ticket for help.
Help us improve! Rate your documentation experience in 5 mins.
Feedback