CVE-2025-11579: rardecode: DoS risk due to unrestricted RAR dictionary sizes

The vulnerability is a denial of service caused by an unrestricted dictionary size in RAR file decoding. The provided patch addresses this by introducing a configurable limit on the dictionary size. The analysis of the patch commit 52fb4e825c936636f251f7e7deded39ab11df9a9 reveals changes in two files: reader.go and volume.go.

In volume.go, a new option maxDictSize is added to the options struct, along with a MaxDictionarySize function to set it. A default value is also provided. This allows the user of the library to control the maximum dictionary size that will be allocated.

In reader.go, the newArchiveFileFrom function is modified to enforce this new limit. The key change is the updated conditional statement:

-		if h.winSize > maxDictSize {
+		if h.winSize > maxDictSize || h.winSize > pr.opt.maxDictSize {
 			return nil, ErrDictionaryTooLarge
 		}

This change shows that newArchiveFileFrom is the function where the dictionary size from the RAR file header (h.winSize) is checked. The vulnerability existed because the previous check was insufficient. By adding the check against pr.opt.maxDictSize, the function now prevents the allocation of excessively large dictionaries.

Therefore, packedFileReader.newArchiveFileFrom is the vulnerable function. When a malicious RAR file is processed, this function would be on the execution path that leads to the out-of-memory error. A runtime profiler would show this function call during an exploit attempt.