The vulnerability is a heap-based buffer overflow in the rust-openssl crate, occurring during encryption with AES key-wrap-with-padding ciphers. The root cause was an incorrect calculation of the required output buffer size in the CipherCtxRef::cipher_update and CipherCtxRef::cipher_update_vec functions.
The original code allocated a buffer of size input.len() + block_size. However, for AES key-wrap-with-padding ciphers (EVP_aes_{128,192,256}_wrap_pad), the OpenSSL library can write up to input.len() + 15 bytes in a single update operation, especially when the input length is not a multiple of 8. This discrepancy meant the allocated buffer could be up to 7 bytes too small, causing the underlying EVP_CipherUpdate call to write past the buffer's boundary, leading to heap corruption.
The patch, identified in commit 257f9b20c001b888986f93579f118fa2a57d4f45, rectifies this by introducing a new function, cipher_update_output_size. This function checks if a wrap-mode cipher is being used. If so, it calculates a correct, larger buffer size using the formula input_len.saturating_add(7) / 8 * 8 + 8. This ensures the buffer is always large enough to accommodate the entire output from OpenSSL, thus preventing the overflow. The vulnerable functions cipher_update and cipher_update_vec were updated to use this new sizing logic.