The vulnerability stems from an incorrect calculation of native memory addresses for Netty ByteBuf objects, specifically when the JVM is configured without access to sun.misc.Unsafe. In this scenario, the library falls back to a method of deriving pointers that fails to account for buffer slicing in pooled allocators.
The root cause is the io.netty.incubator.codec.hpke.boringssl.BoringSSL.memory_address(ByteBuf) function. When a ByteBuf did not expose its memory address directly (i.e., hasMemoryAddress() returned false), this function would create a java.nio.ByteBuffer spanning the entire capacity of the underlying buffer, ignoring the specific offset and length of the ByteBuf slice. This resulted in a base pointer that was incorrect for the slice being used.
This incorrect pointer was then used in cryptographic operations within io.netty.incubator.codec.hpke.boringssl.BoringSSLCryptoOperation.execute, which takes user-provided data in ByteBufs for encryption/decryption. An attacker could send a crafted OHTTP request that triggers these operations, leading to out-of-bounds reads and writes in the native BoringSSL library. This could corrupt or disclose data from other concurrent connections sharing the same memory pool.
The patch rectifies this by removing the flawed memory_address function and introducing readerMemoryAddress and writerMemoryAddress. These new functions correctly calculate the pointer by creating a ByteBuffer view that respects the ByteBuf's specific offset (readerIndex or writerIndex) and length, ensuring all native operations occur within the intended memory boundaries.