产品动态
公告
产品发布记录
--image-gc-high-threshold、--image-gc-low-threshold、--eviction-hard、--eviction-soft 及 --eviction-minimum-reclaim 等参数进行控制以实现磁盘空间的释放。当配置不正确，或者节点上有其它非 K8S 管理的进程在不断写数据到磁盘，将会占用大量空间时将导致磁盘爆满。df 命令，查看 kubelet 和容器运行时所使用的目录是否存在于该磁盘。/var/run/docker：用于存储容器运行状态，通过 dockerd 的 --exec-root 参数指定。/var/lib/docker：用于持久化容器相关的数据。例如，容器镜像、容器可写层数据、容器标准日志输出及通过 docker 创建的 volume 等。Warning FailedCreatePodSandBox 53m kubelet, 172.22.0.44 Failed create pod sandbox: rpc error: code = DeadlineExceeded desc = context deadline exceeded
Warning FailedCreatePodSandBox 2m (x4307 over 16h) kubelet, 10.179.80.31 (combined from similar events): Failed create pod sandbox: rpc error: code = Unknown desc = failed to create a sandbox for pod "apigateway-6dc48bf8b6-l8xrw": Error response from daemon: mkdir /var/lib/docker/aufs/mnt/1f09d6c1c9f24e8daaea5bf33a4230de7dbc758e3b22785e8ee21e3e3d921214-init: no space left on device
Warning Failed 5m1s (x3397 over 17h) kubelet, ip-10-0-151-35.us-west-2.compute.internal (combined from similar events): Error: container create failed: container_linux.go:336: starting container process caused "process_linux.go:399: container init caused \\"rootfs_linux.go:58: mounting \\\\\\"/sys\\\\\\" to rootfs \\\\\\"/var/lib/dockerd/storage/overlay/051e985771cc69f3f699895a1dada9ef6483e912b46a99e004af7bb4852183eb/merged\\\\\\" at \\\\\\"/var/lib/dockerd/storage/overlay/051e985771cc69f3f699895a1dada9ef6483e912b46a99e004af7bb4852183eb/merged/sys\\\\\\" caused \\\\\\"no space left on device\\\\\\"\\""
Normal Killing 39s (x735 over 15h) kubelet, 10.179.80.31 Killing container with id docker://apigateway:Need to kill Pod
/var/lib/kubelet：通过 kubelet 的 --root-dir 参数指定，用于存储插件信息、Pod 相关的状态以及挂载的 volume（例如，emptyDir、ConfigMap及Secret）。Warning UnexpectedAdmissionError 44m kubelet, 172.22.0.44 Update plugin resources failed due to failed to write checkpoint file "kubelet_internal_checkpoint": write /var/lib/kubelet/device-plugins/.728425055: no space left on device, which is unexpected.
$ cd /var/lib/docker/containers$ du -sh * # 找到比较大的目录$ cd dda02c9a7491fa797ab730c1568ba06cba74cecd4e4a82e9d90d00fa11de743c$ cat /dev/null > dda02c9a7491fa797ab730c1568ba06cba74cecd4e4a82e9d90d00fa11de743c-json.log.9 # 删除log文件
cat /dev/null > 方式进行删除，不建议使用 rm。使用 rm 方式删除的文件，不能够被 docker 进程释放掉，该文件所占用的空间也就不会被释放。kubectl drain <node-name>
systemctl restart dockerd# or systemctl restart docker
kubectl uncordon <node-name>
文档反馈