Neat VNC Stack Buffer Overflow Vulnerability in RSA-AES Security Type Handler

A pre-authentication stack buffer overflow vulnerability has been identified in Neat VNC versions prior to 0.9.6. This issue arises in the RSA-AES security type handler, where an unauthenticated remote attacker can send a crafted handshake with an oversized RSA public key. This oversized key causes a buffer overflow in a 1024-byte on-stack buffer while encrypting the server challenge, leading to a server crash and denial-of-service condition. The vulnerability can be exploited by any remote attacker who can reach the VNC listening socket.

Impact

Exploitation of this vulnerability causes a stack buffer overflow, leading to a server crash and denial-of-service condition. This vulnerability is also present in wayvnc and possibly weston.

Reproduction

To reproduce this vulnerability, send a VNC handshake with an oversized RSA public key using security type 5 (RSA-AES) or security type 129 (RSA-AES-256) to a Neat VNC server listening socket. The server will crash, demonstrating the denial-of-service impact of the vulnerability.

Remediation

Users can upgrade to Neat VNC version 0.9.6 or later, where this vulnerability has been fixed.