Linux Kernel Workqueue Performance Degradation Vulnerability

A vulnerability in the Linux kernel's workqueue management can lead to significant performance issues and system crashes. When workqueues are created with the 'WQ_UNBOUND' option, their tasks can be executed on any CPU within the same node. This flexibility, while avoiding CPU contention, can disrupt performance locality. In one observed scenario, a misconfigured process monopolized CPU0, causing delays of up to 30 milliseconds for workqueue tasks scheduled on that CPU. Such delays led to performance degradation, timeouts, and eventual system crashes. A manual test confirmed these issues, showing that without the recent patch, workqueue tasks could be arbitrarily delayed, whereas the patch allows for smoother execution by migrating tasks to less loaded CPUs within the same node.

Impact

The vulnerability causes performance degradation, timeouts, and system crashes by allowing workqueue tasks to be delayed on overloaded CPUs.

Reproduction

The vulnerability can be reproduced by creating a workqueue with the 'WQ_UNBOUND' option and then binding a high-priority process to a specific CPU, such as CPU0. This process will consume most of the CPU's resources, leaving less than 0.5% available for workqueue tasks. The IDPF driver can be manipulated to prefer CPU0, forcing workqueue tasks to experience significant delays. Without the patch, these delays can be as high as 30 milliseconds, but with the patch applied, the maximum delay reduces to just 6 microseconds.

Remediation

Users can apply the latest patch available in the Linux kernel to address this vulnerability.