Linux Kernel Memory Cgroup Soft Lockup Vulnerability During Out-Of-Memory Process Handling

A soft lockup vulnerability has been identified in the Linux kernel's memory cgroup management, specifically during the out-of-memory (OOM) process. When approximately 56,000 tasks were in the OOM cgroup, the system experienced a soft lockup, with CPU#2 reportedly stuck for 23 seconds. This issue arose because the OOM process took a long time to traverse the large number of tasks, leading to a soft lockup. The vulnerability has been addressed by modifying the task scanning process to include periodic rescheduling, and by updating the global OOM handling to prevent soft lockups.

Impact

The vulnerability caused a soft lockup, where the CPU was stuck for an extended period, disrupting normal process execution.