Linux Kernel Control and User Packet Handling Deadlock Vulnerability in n_gsm TTY Layer

A vulnerability in the Linux kernel's TTY layer, specifically within the n_gsm protocol, has been addressed. The issue arose because the implementation tightly coupled control and user packet processing, leading to deadlocks during data transmission, particularly when the line discipline (ldisc) was congested. This congestion caused data channels to starve the control channel under heavy load. The vulnerability has been resolved by introducing a separate control channel queue to manage packets more effectively, preventing timeouts and link hangups during ldisc congestion. The new control queue is processed with higher priority than user data, ensuring better performance under heavy transmission loads. Additionally, the fix includes a mechanism to clear packets from the queue associated with closed Data Link Connection Identifiers (DLCIs), preventing data from being sent to inactive channels.

Impact

The vulnerability could cause a deadlock in the TTY layer's n_gsm protocol, leading to transmission failures and potential link hangups during periods of high data load.

Reproduction

The vulnerability can be reproduced by transmitting data over a TTY connection using the n_gsm protocol while the line discipline is congested. This will create a deadlock situation, as new control and user packets will be queued but not processed, especially under high transmission loads.

Remediation

Users can upgrade to the latest version of the Linux kernel where this vulnerability has been fixed.