The vulnerability is an out-of-bounds read from the h_subpel_filters_outer array in libavcodec/ppc/vp8dsp_altivec.c. The commit 09e6840cf7a3ee07a73c3ae88a020bf27ca1a667 fixes this by increasing the array's size from 3 to 4. The commit message explains that h_subpel_filters_inner[i] and h_subpel_filters_outer[i / 2] are used together, and h_subpel_filters_inner allows i up to 6. This means h_subpel_filters_outer needs to support an index of 6 / 2 = 3. Before the patch, with a size of 3, accessing index 3 was out-of-bounds.
The array h_subpel_filters_outer is accessed within the LOAD_H_SUBPEL_FILTER(i) macro as h_subpel_filters_outer[(i)>>1]. This macro is used by the static av_always_inline function put_vp8_epel_h_altivec_core(..., int mx, ...), where the argument to the macro is mx-1.
Since put_vp8_epel_h_altivec_core is inlined, the functions that call it effectively contain the vulnerable code. These callers are generated by the EPEL_FUNCS macro (e.g., put_vp8_epel16_h6_altivec) and are av_noinline, making them likely to appear in runtime profiles.
Additionally, functions generated by the EPEL_HV macro (e.g., put_vp8_epel16_h6v6_altivec) call these EPEL_FUNCS functions, and thus also lead to the execution of the vulnerable code. All these functions are static C functions within the specified file path. The vulnerability is triggered when the mx parameter to these functions (or derived from it) results in (mx-1) >> 1 being 3.
Ongoing coverage of React2Shell