The vulnerability is caused by unbounded memory allocation in the Mach-O binary parsing logic of the Quill library. The root cause is the failure to validate several size and count fields (DataSize, DataOffset, Count, Length) read from the LC_CODE_SIGNATURE load command and its nested code signing structures. An attacker can craft a minimal Mach-O file with extremely large values in these fields. When Quill processes this file, it uses these unvalidated values to allocate memory buffers. This leads to excessive memory allocation, causing a denial of service by crashing the host process.
The patch addresses this by introducing several validation checks. It defines maximum allowable values for the superblob size, blob count, and individual blob lengths. It also adds a new function, validateDataRange, to ensure that the data ranges specified in the binary (offset and size) are actually within the bounds of the file. These checks are applied in all functions that parse the code signing data, including CDBytes, CMSBlobBytes, HashPages, and RemoveSigningContent, effectively mitigating the vulnerability.