当前位置：
  1. 首页
  2. 互联网
  3. 正文
本文介绍: ulimit 限制 fd 总数，限制级别进程，可对所有用户生效；ulimit 限制线程总数，限制级别用户（uid)，限制同一个 uid 下所有线程/进程数，对于 root 账号无效；对于目前线上情况，有较小的概率因 ulimit 限制导致 fork 失败，如同一个宿主机中有多个 work 容器且基础镜像相同（即 uid 相同），若一个容器线程泄露，由于 ulimit 限制会影响其他容器正常运行。

一、背景

二、实验对比

① 环境

② ulimit

三、文件描述符限制

RLIMIT_NOFILE
              This specifies a value one greater than the maximum file
              descriptor number that can be opened by this process.
              Attempts (open(2), pipe(2), dup(2), etc.)  to exceed this
              limit yield the error EMFILE.  (Historically, this limit was
              named RLIMIT_OFILE on BSD.)

              Since Linux 4.5, this limit also defines the maximum number of
              file descriptors that an unprivileged process (one without the
              CAP_SYS_RESOURCE capability) may have "in flight" to other
              processes, by being passed across UNIX domain sockets.  This
              limit applies to the sendmsg(2) system call.  For further
              details, see unix(7).

$ docker run -d --ulimit nofile=100:200  cr.d.xiaomi.net/containercloud/alpine:webtool top

/ # ulimit -a
-f: file size (blocks)             unlimited
-t: cpu time (seconds)             unlimited
-d: data seg size (kb)             unlimited
-s: stack size (kb)                8192
-c: core file size (blocks)        unlimited
-m: resident set size (kb)         unlimited
-l: locked memory (kb)             64
-p: processes                      unlimited
-n: file descriptors               100
-v: address space (kb)             unlimited
-w: locks                          unlimited
-e: scheduling priority            0
-r: real-time priority             0

/ # ab -n 1000000 -c 90 http://61.135.169.125:80/ &
/ # lsof | wc -l 
108
/ # lsof | grep -c ab
94

/ #  ab -n 1000000 -c 100 http://61.135.169.125:80/
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking 61.135.169.125 (be patient)
socket: No file descriptors available (24)

四、线程限制

RLIMIT_NPROC
              This is a limit on the number of extant process (or, more pre‐
              cisely on Linux, threads) for the real user ID of the calling
              process.  So long as the current number of processes belonging
              to this process's real user ID is greater than or equal to
              this limit, fork(2) fails with the error EAGAIN.

              The RLIMIT_NPROC limit is not enforced for processes that have
              either the CAP_SYS_ADMIN or the CAP_SYS_RESOURCE capability.

五、容器 uid

$ docker run -d  cr.d.xiaomi.net/containercloud/alpine:webtool top

$ ps -ef |grep top
root      4096  4080  0 15:01 ?        00:00:01 top

/ # id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel),11(floppy),20(dialout),26(tape),27(video)
/ # su operator
/ $ id
uid=11(operator) gid=0(root) groups=0(root)
/ $ sleep 100
$ ps -ef |grep 'sleep 100'
app      19302 19297  0 16:39 pts/0    00:00:00 sleep 100
$ cat /etc/passwd | grep app
app❌11:0::/home/app:

六、验证不同用户下 ulimit 的限制

$ docker run -d --ulimit nproc=10:20  cr.d.xiaomi.net/containercloud/alpine:webtool top
  • 进入容器查看， fd soft 限制为 100 个：
/ # ulimit -a
-f: file size (blocks)             unlimited
-t: cpu time (seconds)             unlimited
-d: data seg size (kb)             unlimited
-s: stack size (kb)                8192
-c: core file size (blocks)        unlimited
-m: resident set size (kb)         unlimited
-l: locked memory (kb)             64
-p: processes                      10
-n: file descriptors               1048576
-v: address space (kb)             unlimited
-w: locks                          unlimited
-e: scheduling priority            0
-r: real-time priority             0

/ # for i in `seq 30`;do sleep 100 &amp;; done
/ # ps | wc -l 
36

/ # su operator

# 启动多个进程，到第11个进程无法进行fork
/ $ for i in `seq 8`; do
> sleep 100 &amp;
> done
/ $ sleep 100 &amp;
/ $ sleep 100 &amp;
sh: can't fork: Resource temporarily unavailable
  • root 下查看：
/ # ps -ef | grep operator
   79 operator  0:00 sh
   99 operator  0:00 sleep 100
  100 operator  0:00 sleep 100
  101 operator  0:00 sleep 100
  102 operator  0:00 sleep 100
  103 operator  0:00 sleep 100
  104 operator  0:00 sleep 100
  105 operator  0:00 sleep 100
  106 operator  0:00 sleep 100
  107 operator  0:00 sleep 100
  109 root      0:00 grep operator
/ # ps -ef | grep operator| wc -l
10

七、验证 ulimit 在不同容器相同 uid 下的限制

$ docker run -d --ulimit nproc=3:3 --name nproc1 -u operator  cr.d.xiaomi.net/containercloud/alpine:webtool top
eeb1551bf757ad4f112c61cc48d7cbe959185f65109e4b44f28085f246043e65
$ docker run -d --ulimit nproc=3:3 --name nproc2 -u operator  cr.d.xiaomi.net/containercloud/alpine:webtool top
42ff29844565a9cb3af2c8dd560308b1f31306041d3dbd929011d65f1848a262
$ docker run -d --ulimit nproc=3:3 --name nproc3 -u operator  cr.d.xiaomi.net/containercloud/alpine:webtool top
b7c9b469e73f969d922841dd77265467959eda28ed06301af8bf83bcf18e8c23
$ docker run -d --ulimit nproc=3:3 --name nproc4 -u operator  cr.d.xiaomi.net/containercloud/alpine:webtool top
b49d8bb58757c88f69903059af2ee7e2a6cc2fa5774bc531941194c52edfd763
$
$ docker ps -a |grep nproc
b49d8bb58757        cr.d.xiaomi.net/containercloud/alpine:webtool      "top"                    16 seconds ago      Exited (1) 15 seconds ago                               nproc4
b7c9b469e73f        cr.d.xiaomi.net/containercloud/alpine:webtool      "top"                    23 seconds ago      Up 22 seconds                                           nproc3
42ff29844565        cr.d.xiaomi.net/containercloud/alpine:webtool      "top"                    31 seconds ago      Up 29 seconds                                           nproc2
eeb1551bf757        cr.d.xiaomi.net/containercloud/alpine:webtool      "top"                    38 seconds ago      Up 36 seconds                                           nproc1

八、总结

① ulimit

② cgroup

[root@node01 ~]# ps -ef |grep kubelet
root     18735     1 14 11:19 ?        00:53:28 ./kubelet --v=1 --address=0.0.0.0 --feature-gates=SupportPodPidsLimit=true --pod-max-pids=150 --allow-privileged=true --pod-infra-container-image=cr.d.xiaomi.net/kubernetes/pause-amd64:3.1 --root-dir=/home/kubelet --node-status-update-frequency=5s --kubeconfig=/home/xbox/kubelet/conf/kubelet-kubeconfig --fail-swap-on=false --max-pods=254 --runtime-cgroups=/systemd/system.slice/frigga.service --kubelet-cgroups=/systemd/system.slice/frigga.service --make-iptables-util-chains=false
  • 在 pod 中起测试线程，root 下起 100 个线程：
/ # for i in `seq 100`; do
> sleep 1000 &amp;
> done
/ # ps | wc -l
106
  • operator 下，创建线程受到限制，系统最多只能创建 150 个：
/ # su operator
/ $ 
/ $ for i in `seq 100`; do
> sleep 1000 &
> done
sh: can't fork: Resource temporarily unavailable
/ $ ps | wc -l
150

[root@node01 ~]# cat /sys/fs/cgroup/pids/kubepods/besteffort/pod8b61d4de-a7ad-11e9-b5b9-246e96ad0900/pids.current 
150
[root@node01 ~]# cat /sys/fs/cgroup/pids/kubepods/besteffort/pod8b61d4de-a7ad-11e9-b5b9-246e96ad0900/pids.max 
150

③ limits.conf/sysctl.conf

$ docker run -d --ulimit nofile=100:200 cr.d.xiaomi.net/containercloud/alpine:webtool top
cb1250c8fd217258da51c6818fa2ce2e2f6e35bf1d52648f1f432e6ce579cf0d
$ docker exec -it cb1250c sh

/ # ulimit -a
-f: file size (blocks)             unlimited
-t: cpu time (seconds)             unlimited
-d: data seg size (kb)             unlimited
-s: stack size (kb)                8192
-c: core file size (blocks)        unlimited
-m: resident set size (kb)         unlimited
-l: locked memory (kb)             64
-p: processes                      unlimited
-n: file descriptors               100
-v: address space (kb)             unlimited
-w: locks                          unlimited
-e: scheduling priority            0
-r: real-time priority             0
/ # 
/ # echo 10 > /proc/sys/kernel/pid_max
sh: can't create /proc/sys/kernel/pid_max: Read-only file system
/ # echo 10 > /proc/sys/kernel/pid_max
sh: can't create /proc/sys/kernel/pid_max: Read-only file system
/ # echo "fs.file-max=5" >> /etc/sysctl.conf
/ # sysctl -p
sysctl: error setting key 'fs.file-max': Read-only file system

$ cat /proc/sys/kernel/pid_max
32768
$ docker run -d -- --ulimit nofile=100:200 cr.d.xiaomi.net/containercloud/alpine:webtool top
$ docker exec -it pedantic_vaughan sh
/ # cat /proc/sys/kernel/pid_max
32768
/ # echo 50000 > /proc/sys/kernel/pid_max
/ # cat /proc/sys/kernel/pid_max
50000
/ # exit
$ cat /proc/sys/kernel/pid_max
50000 # 宿主机的文件也变成50000

在这里插入图片描述

原文地址:https://blog.csdn.net/Forever_wj/article/details/134783137

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。

如若转载，请注明出处：http://www.7code.cn/show_42728.html

如若内容造成侵权/违法违规/事实不符，请联系代码007邮箱：suwngjj01@126.com进行投诉反馈，一经查实，立即删除！

上一篇 机器学习——logistic回归
下一篇 Android：BackStackRecord

相关文章

发表回复

您的电子邮箱地址不会被公开。 必填项已用 * 标注

插入图片
浏览器会保存昵称、邮箱和网站cookies信息，下次评论时使用。