是这样的，今天发现 k8s 有 2 台 master 节点 CPU 使用率有尖峰（ k8s 1.11 Linux 3.10 1.13.1 ），由于没有当时进程状态快照，只能用 top 每秒抓一次进程扔日志里。果然过了一段时间，发现了几个叫 docker-containerd-current 的进程 cpu 飙升导致 CPU 使用率高达 90%，但是我们通过 ps pgrep 等等手段都查不到这个进程了，所以考虑是不是当时突然启动了这些进程然后又死了，后来想从 /proc 里查有没有啥信息，奇怪的事发生了：ll -arth /proc|grep $PID 通过 pid 号对应文件夹查不到这个进程，但是 cd $PID 居然可以进到进程的文件夹里，然后我不死心，退回到 /proc 里又肉眼看了一遍文件夹，还是没找到。进到 /proc/$PID 内查看了 exe 对应的二进制文件，然后通过 ps -ef|grep $EXE_FILE 发现对应的进程启动时间还是去年。

这里的问题就是 /proc 下面的以 pid 命名的文件夹不应该是正在运行着的进程的么？进程确实死了，通过文件名也查不到了，但是为什么可以进到一个根本不存在的文件夹呢？

重点是图里面圈中的部分。

ls 显示目录的参数是-d

不用加这个参数也可以看文件夹啊

ll 是什么的 alias

ls -l

不加也可以，输出太多更乱

但是问题是查不到文件夹为啥可以进去，而且进程都死了，不应该在 proc 下了啊

具体原因不清楚，可以用 lsof -p 14546 看下有没有结果

看过 没有

可能是进程被隐藏了。

还有这种操作？那么有什么方法能看到被隐藏的进程吗？


你看一下现在 procfs 的 mount options 是什么

https://unix.stackexchange.com/questions/534596/hiding-processes-on-unix-linux


如果没有什么特殊的 mount options，那说明你这个不是 pid，而是某个 tid，你需要找到它的主进程再看。
/proc/ 是不会出现在 getdents 调用中，所以不会被 ls 看到

感谢兄弟，学到一个新知识。

老哥 我看有 mountinfo mounts mountstats 截图里是 mountstats 的输出，tid 是线程 id 吗？
device rootfs mounted on / with fstype rootfsdevice sysfs mounted on /sys with fstype sysfs
device proc mounted on /proc with fstype procdevice devtmpfs mounted on /dev with fstype devtmpfs
device securityfs mounted on /sys/kernel/security with fstype securityfsdevice tmpfs mounted on /dev/shm with fstype tmpfsdevice devpts mounted on /dev/pts with fstype devptsdevice tmpfs mounted on /run with fstype tmpfs
device tmpfs mounted on /sys/fs/cgroup with fstype tmpfsdevice cgroup mounted on /sys/fs/cgroup/systemd with fstype cgroupdevice pstore mounted on /sys/fs/pstore with fstype pstoredevice cgroup mounted on /sys/fs/cgroup/pids with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/blkio with fstype cgroupdevice cgroup mounted on /sys/fs/cgroup/perf_event with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/freezer with fstype cgroupdevice cgroup mounted on /sys/fs/cgroup/net_cls,net_prio with fstype cgroupdevice cgroup mounted on /sys/fs/cgroup/devices with fstype cgroupdevice cgroup mounted on /sys/fs/cgroup/cpu,cpuacct with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/hugetlb with fstype cgroup
device cgroup mounted on /sys/fs/cgroup/cpuset with fstype cgroupdevice cgroup mounted on /sys/fs/cgroup/memory with fstype cgroup
device configfs mounted on /sys/kernel/config with fstype configfsdevice /dev/mapper/vg_root-lv_root mounted on / with fstype xfsdevice selinuxfs mounted on /sys/fs/selinux with fstype selinuxfsdevice systemd-1 mounted on /proc/sys/fs/binfmt_misc with fstype autofs
device hugetlbfs mounted on /dev/hugepages with fstype hugetlbfsdevice debugfs mounted on /sys/kernel/debug with fstype debugfs
device mqueue mounted on /dev/mqueue with fstype mqueue
device /dev/sda1 mounted on /boot with fstype xfsdevice /dev/mapper/vg_root-lv_var mounted on /var with fstype xfsdevice /dev/mapper/vg_root-lv_opt mounted on /opt with fstype xfsdevice /dev/mapper/vg_app-lv_DATA mounted on /DATA with fstype xfs
device /dev/mapper/vg_root-lv_tmp mounted on /tmp with fstype xfs
device /dev/mapper/vg_root-lv_home mounted on /home with fstype xfs
device sunrpc mounted on /var/lib/nfs/rpc_pipefs with fstype rpc_pipefs
device /dev/mapper/vg_root-lv_var mounted on /var/lib/docker/containers with fstype xfs
device /dev/mapper/vg_root-lv_var mounted on /var/lib/docker/overlay2 with fstype xfs
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/206f7c4c-52cd-11ea-9f39-0050569a04bf/volumes/kubernetes.io~secret/sdn-token-d66w8 with fstype tmpfs
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/e3695fa7-2dce-11ea-97bb-005056a99505/volumes/kubernetes.io~secret/sdn-token-d66w8 with fstype tmpfs
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/5abfd92e-2dc8-11ea-97bb-005056a99505/volumes/kubernetes.io~secret/sync-token-tr7f8 with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/c47caa96460e233c36b6790b1ca4ea450dee1d297f9e40bf199c6ab83bb1e690/merged with fstype overlay
device overlay mounted on /var/lib/docker/overlay2/edb026a3c373d3ddbd064b9d8e997ccb441d56eb883f19237374bdd839cd876a/merged with fstype overlay
device proc mounted on /run/docker/netns/default with fstype proc
device shm mounted on /var/lib/docker/containers/6d3e50e348e7bc4608b568fd43ad63670c57f0c53de71f446ba17e8ec5e47f4e/shm with fstype tmpfs
device shm mounted on /var/lib/docker/containers/f2c824ce425cfe15dda053af70d4fd02d64afdae9367ae78bcd6b5758338182c/shm with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/60fffaf1c2c9cbe0d12504a67a0b603c9bdbf8b0f6322dc30b2befa9cbb87438/merged with fstype overlay
device overlay mounted on /var/lib/docker/overlay2/0045f1ad6a6556324b91b80892dcc4951ad32ec8dfe153b7c866b942fae74843/merged with fstype overlay
device overlay mounted on /var/lib/docker/overlay2/ec0204696ace63e3c1eb69bd81e029c13d7c1320b16b561ddceb3c56f4ae9172/merged with fstype overlay
device shm mounted on /var/lib/docker/containers/a774eddee040f246b9dea20a33781a1b8cb80a3bc668690d66baec33a0f56a1d/shm with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/cfcfc406430dc094d2b2a7084f38ea44b1eb2200282ce5fa432d46c9fee9d1f6/merged with fstype overlay
device shm mounted on /var/lib/docker/containers/6d7a77c6a7dade24e619f63aa813962ae003e52f15dc146141af3c362564fd27/shm with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/52230e78851c2d66d058e1fda52b3cb8b3493a86804354d234fe64895b3aed56/merged with fstype overlay
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/a684975d-9ef2-11ea-90f1-005056a9a751/volumes/kubernetes.io~secret/node-exporter-token-mttqc with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/c07d51282d5db3eeaa6966bdb277d34fdab0707900b34822b5bb85c6f3dddc84/merged with fstype overlay
device shm mounted on /var/lib/docker/containers/14c0c7e261e57a3eac08de62225b5b33fe8a11d231b13e34ddbdec8231e98d24/shm with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/7c510eedb0ebb50ace710478ceca3037011a99a24ded9b2862d2700064782b22/merged with fstype overlay
device overlay mounted on /var/lib/docker/overlay2/be29c4a1f9bbd28181851f27df38bf277a1984595490599db5d4297de8d0847f/merged with fstype overlay
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/42a3edd4-02be-11eb-854a-005056a958e5/volumes/kubernetes.io~secret/oauth-config with fstype tmpfs
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/42a3edd4-02be-11eb-854a-005056a958e5/volumes/kubernetes.io~secret/serving-cert with fstype tmpfs
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/42a3edd4-02be-11eb-854a-005056a958e5/volumes/kubernetes.io~secret/default-token-mlq5l with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/2a50c3d9829ae039bbe3afb1108e2b131aca83febf40070b0423761aee576354/merged with fstype overlay
device shm mounted on /var/lib/docker/containers/4b04ef977813ab3b24704d433d09d56fd069ee9103a32c5d041a3a072eab4663/shm with fstype tmpfs
device proc mounted on /run/docker/netns/4dd48aec066f with fstype proc
device overlay mounted on /var/lib/docker/overlay2/e1ee93593bab0e24a711c4ce871534fefe651e8be5ee11cf64f544e0b3e2e86f/merged with fstype overlay
device binfmt_misc mounted on /proc/sys/fs/binfmt_misc with fstype binfmt_misc
device overlay mounted on /var/lib/docker/overlay2/e04f7635399c36249c8e9b90d5619ba593b7633275d1b2c2363d3c93d0e2f55b/merged with fstype overlay
device overlay mounted on /var/lib/docker/overlay2/0be6198515753a8a97b51d078ac64e1153325342221e8d77a956eecabd448d12/merged with fstype overlay
device shm mounted on /var/lib/docker/containers/f54abb6462617d5910b8d41e5fa4c131be18aa59d4d227040e3f3e2ecd69d70a/shm with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/a177d06ddec1fb54a0b04725fabe944293ca72eb4aa0bdbf4e918f828d089466/merged with fstype overlay
device overlay mounted on /var/lib/docker/overlay2/0af1e5fe64c765d3e0a4d574b725829a6b6e3170b929ede21cdd5b5cb97b0fc4/merged with fstype overlay
device shm mounted on /var/lib/docker/containers/284f17f11532b9750a1f15adb3c2361796c42c4e0afc220c49c0cd394c477299/shm with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/ce6e3d2eb2879de645014da2be3650c7f172634131d08718304ef106a3b4ea66/merged with fstype overlay
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/37a179b0-690c-11eb-b1fb-005056a9a751/volumes/kubernetes.io~secret/serving-cert with fstype tmpfs
device tmpfs mounted on /var/lib/origin/openshift.local.volumes/pods/37a179b0-690c-11eb-b1fb-005056a9a751/volumes/kubernetes.io~secret/webconsole-token-spxpx with fstype tmpfs
device overlay mounted on /var/lib/docker/overlay2/188242c0ee6187645f21d2f0195615af2d450337a461c4c197367bfaf794d5ed/merged with fstype overlay
device shm mounted on /var/lib/docker/containers/5b5fb40679472f61b4b5f37caf20666291fdb38f04648710dbd23a3e952c0845/shm with fstype tmpfs
device proc mounted on /run/docker/netns/feff0fd124b3 with fstype proc
device overlay mounted on /var/lib/docker/overlay2/d6d7732f83a71c8c3f33a257bab98a140b5099e0ba2b1bb789c057520cf75d95/merged with fstype overlay
device tmpfs mounted on /run/user/0 with fstype tmpfs

错了，不是截图是是文本