CVE-2023-5678: Issue summary: Generating excessively long X9.42 DH keys or checking excessively long X9.42 DH...

The vulnerability CVE-2023-5678 in OpenSSL relates to potential Denial of Service attacks due to excessively long computation times when handling X9.42 DH keys or parameters with overly large P or Q values. The analysis of the provided patches (commits 34efaef6c103d636ab507a0cc34dca4d3aecc055, ddeb4b6c6d527e54ce9a99cba785c0f7776e54b6, and db925ae2e65d0d925adef429afc37f75bd1c2017) and the vulnerability description allowed for the identification of the core functions where the necessary input validation was missing.

1. DH_check_pub_key: The vulnerability description explicitly states this function was missing checks. The patches confirm the addition of checks for both P (modulus) and Q sizes. The absence of these checks meant that untrusted, excessively large parameters could be processed, leading to DoS.

2. generate_key (static function): The public function DH_generate_key relies on this internal static function. The description notes DH_generate_key didn't check for an excessively large Q. The patches show the Q size check was added to this static generate_key function.

3. ossl_dh_compute_key (and its older counterpart compute_key): Patches also reveal that this internal computation function was missing a Q size check. This function is involved in DH shared secret computation and could be called by other affected public APIs like DH_check_pub_key_ex. Its lack of validation contributed to the overall vulnerability.

These functions are identified as vulnerable because they are the specific locations in the codebase where the crucial size validation checks for DH parameters were absent prior to the patches. Public API functions like DH_generate_key(), DH_check_pub_key_ex(), EVP_PKEY_public_check(), and EVP_PKEY_generate() are affected because they directly or indirectly call these lower-level functions that contained the flawed logic.