Linux Kernel GEM Plane Dimension Calculation Vulnerability in Framebuffer Initialization

A vulnerability exists in the Linux kernel's Graphics Execution Manager (GEM) framebuffer handling. The issue arises in the 'drm_gem_fb_init_with_funcs()' function, which calculates sub-sampled plane dimensions using standard integer division. This method can lead to incorrect dimension values, particularly with certain pixel formats like NV12. The problem is exacerbated by the fact that the 'framebuffer_check()' function uses a different method that rounds up dimensions, creating a mismatch. As a result, the GEM object size validation can be bypassed, allowing the GPU to access memory beyond the bounds of the object, potentially leading to memory corruption.

Impact

Exploitation of this vulnerability can cause memory corruption by allowing the GPU to read or write outside the allocated bounds of a GEM object, particularly when accessing chroma planes.

Reproduction

The vulnerability can be reproduced by creating a GEM-backed framebuffer with a pixel format that requires vertical subsampling, such as NV12. When the framebuffer is initialized with a height of one pixel, the improper calculation of the plane dimensions will cause the GEM size validation to fail, allowing a tiny GEM object to pass through the size guard. Subsequently, when the GPU accesses the chroma plane, it will read or write memory beyond the object's allocated bounds, causing memory corruption.

Remediation

The vulnerability has been addressed by modifying the 'drm_gem_fb_init_with_funcs()' function to use 'drm_format_info_plane_width()' and 'drm_format_info_plane_height()' for dimension calculations. This change ensures that the dimensions are rounded up correctly and match the values used in the 'framebuffer_check()' function, preventing the size validation bypass. Users should upgrade to the latest stable version of the Linux kernel where this fix has been applied.