The vulnerability is a classic 'decode-after-check' path traversal flaw in NLTK's data loading mechanism. The root cause is that the security check to prevent path traversal was performed on the raw, URL-encoded resource string. Malicious input could use percent-encoded characters (e.g., %2f for /, %2e for .) to bypass this check. The url2pathname() function, used later in the process, would decode these characters, reassembling the malicious path and allowing an attacker to read arbitrary files on the local filesystem.
The analysis of the patch commit 7df10e6efa0417145e370aa42d8342faa6d244b8 confirms this. The fix involves introducing a new helper function, _assert_no_encoded_bypass, which explicitly decodes the resource name using urllib.parse.unquote and then applies the security regex (_UNSAFE_NO_PROTOCOL_RE) to the decoded string.
This new check is applied in two key vulnerable functions:
nltk.data.find(): This function, which is responsible for locating the data on the filesystem, now validates both the raw and decoded resource name.
nltk.data.normalize_resource_url(): This function, which processes nltk: scheme URLs, also incorporates the new check to prevent bypasses at the normalization stage.
The primary user-facing function and entry point for exploitation is nltk.data.load(), as demonstrated in the provided proof-of-concept. This function internally calls find(), triggering the vulnerable logic. Therefore, any of these three functions (load, find, normalize_resource_url) would likely appear in a runtime profile or stack trace during an exploit attempt. The fix ensures that any path traversal attempts, whether literal or URL-encoded, are caught and rejected.