Linux Kernel TCP Window Advertising Vulnerability Under Memory Pressure

A vulnerability in the Linux kernel's TCP implementation has been identified, specifically in how the protocol handles window advertising during extreme memory pressure. When a socket endpoint temporarily advertises a zero-sized window, this information is not retained in the socket's data, as it is viewed as a temporary state that shouldn't affect future calculations. However, if the window size stalls at a low value, the algorithm responsible for selecting a new window size may consistently fail to advertise a non-zero window, even after memory is freed. This discrepancy can cause the peer to stop sending data, leading to a stall in communication. The issue has been observed on the iperf3 server side, using the default settings of the Fedora 40 kernel, without any special socket options.

Impact

The vulnerability can cause a denial of service on the TCP connection, where the peer stops sending data due to the incorrect window size advertisement, leading to a stall in communication.

Reproduction

The vulnerability can be reproduced by using the iperf3 tool with the 'pasta' protocol splicer, under conditions of extreme memory pressure. This will cause the TCP socket to advertise a zero-sized window, which is not properly managed, leading to a situation where the window size does not get updated correctly. As a result, the peer will perceive the window as closed and will not send additional data, causing a stall in the connection.