The vulnerability lies in the recursive nature of the JSON deserializer in serde-json-wasm. When parsing deeply nested JSON objects or arrays, the deserializer functions would call themselves or other deserializer functions without any limit on the recursion depth. This leads to a stack overflow, causing a denial of service.
The patch addresses this by introducing a recursion limit. A remaining_depth counter is added to the Deserializer struct, initialized to a safe limit (128). A macro, check_recursion!, is then used to wrap the recursive calls within the deserialization logic. This macro decrements the counter before a recursive call and increments it after, returning a RecursionLimitExceeded error if the counter reaches zero.
The vulnerable functions are the methods of the Deserializer that handle the deserialization of recursive structures (arrays and maps), namely deserialize_any, deserialize_seq, deserialize_map, and deserialize_enum. The patch files clearly show the addition of the check_recursion! macro in these functions, which is direct evidence of where the vulnerability was addressed. Any of these functions could appear in a stack trace during the exploitation of this vulnerability when a malicious, deeply nested JSON is provided as input.