Skip to content

Harden OpenSSL crypto error checking and resource management#7817

Merged
maxtropets merged 11 commits into
mainfrom
copilot/add-check1-to-crypto-files
Apr 16, 2026
Merged

Harden OpenSSL crypto error checking and resource management#7817
maxtropets merged 11 commits into
mainfrom
copilot/add-check1-to-crypto-files

Conversation

Copilot AI commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Harden OpenSSL error checking and resource management in ccf::crypto:

  • CHECK1 (openssl_wrappers.h): Threw only when error queue was non-empty. Now throws unconditionally on rc != 1; error message includes rc, ec, and best-effort error string.

  • CHECK0 removed: Ambiguous semantics replaced by CHECKPOSITIVE(). Removal noted in CHANGELOG (Harden OpenSSL crypto error checking and resource management #7817).

  • CHECKPOSITIVE (new): For OpenSSL APIs returning positive-on-success. Includes ERR_get_error() / error_string() and rc/ec in diagnostics.

  • RSA OAEP params (rsa_key_pair.cpp, rsa_public_key.cpp): EVP_PKEY_CTX_set_rsa_padding, _set_rsa_oaep_md, _set_rsa_mgf1_md return values were unchecked. Wrapped with CHECKPOSITIVE().

  • HKDF context leak (hash.cpp): Raw EVP_PKEY_CTX* leaked on throw. Replaced with Unique_EVP_PKEY_CTX RAII wrapper.

  • Wrong check macro (ec_key_pair.cpp): X509_REQ_sign returns signature size (positive) on success, 0 on failure — corrected from CHECK1 to CHECKPOSITIVE.

  • salt_length overflow (rsa_key_pair.cpp, rsa_public_key.cpp): size_t salt_length was passed directly to EVP_PKEY_CTX_set_rsa_pss_saltlen(int) without range checking. Added validation throwing std::invalid_argument if salt_length > INT_MAX.

OpenSSL doc links for CHECK1CHECKPOSITIVE replacements:

EVP_PKEY_CTX_set_rsa_padding / set_rsa_oaep_md / set_rsa_mgf1_md / set_rsa_pss_saltlen / set_signature_md
"return a positive value for success and 0 or a negative value for failure"
https://docs.openssl.org/master/man3/EVP_PKEY_CTX_ctrl/#return-values

EVP_PKEY_CTX_set_hkdf_md / set1_hkdf_salt / set1_hkdf_key / add1_hkdf_info
Same page as above — all are EVP_PKEY_CTX_ctrl wrappers
https://docs.openssl.org/master/man3/EVP_PKEY_CTX_ctrl/#return-values

sk_X509_push (→ OPENSSL_sk_push)
"returns the total number of elements in the stack after the push, or 0 on error"
https://docs.openssl.org/master/man3/OPENSSL_sk_push/#return-values

X509_REQ_sign
"returns the size of the signature in bytes [...] or 0 on failure"
https://docs.openssl.org/master/man3/X509_sign/#return-values

i2d_ECDSA_SIG
"returns the length of the DER encoded signature [...] or negative value on error"
https://docs.openssl.org/master/man3/i2d_ECDSA_SIG/#return-values

@maxtropets maxtropets marked this pull request as ready for review April 14, 2026 12:45
@maxtropets maxtropets requested a review from a team as a code owner April 14, 2026 12:45
Copilot AI review requested due to automatic review settings April 14, 2026 12:45
@maxtropets maxtropets added the run-long-test Run Long Test job label Apr 14, 2026
Comment thread src/crypto/openssl/hash.cpp
Comment thread src/crypto/openssl/rsa_public_key.cpp Outdated
Copilot AI requested a review from maxtropets April 14, 2026 13:03

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@maxtropets maxtropets marked this pull request as draft April 14, 2026 13:34
@maxtropets

Copy link
Copy Markdown
Collaborator

@copilot based on the last commit I've done, update the PR title and description if needed

Also make them as concise as possible w/o losing any useful info

Copilot AI commented Apr 14, 2026

Copy link
Copy Markdown
Contributor Author

@copilot based on the last commit I've done, update the PR title and description if needed

Also make them as concise as possible w/o losing any useful info

Updated the PR description to cover all four changes including the CHECK0 hardening and the X509_REQ_sign CHECK1CHECK0 fix from 962d640. Kept the existing title — it still accurately covers everything.

@maxtropets maxtropets marked this pull request as ready for review April 14, 2026 14:16
Comment thread src/crypto/openssl/rsa_key_pair.cpp Outdated

@achamayou achamayou left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Goes in the right direction but please see and fix comment.

@maxtropets maxtropets requested a review from Copilot April 15, 2026 10:13
@maxtropets maxtropets enabled auto-merge (squash) April 15, 2026 10:18

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Comment thread include/ccf/crypto/openssl/openssl_wrappers.h Outdated
Comment thread include/ccf/crypto/openssl/openssl_wrappers.h
Comment thread include/ccf/crypto/openssl/openssl_wrappers.h
Comment thread src/crypto/openssl/rsa_public_key.cpp
…GELOG entry for CHECK0 removal

Agent-Logs-Url: https://github.com/microsoft/CCF/sessions/4d6d118d-4cf4-46fb-93c6-3fd4806cf748

Co-authored-by: maxtropets <16566519+maxtropets@users.noreply.github.com>

Copilot AI commented Apr 15, 2026

Copy link
Copy Markdown
Contributor Author

@copilot based on the last commit I've done, update the PR title and description if needed

Also make them as concise as possible w/o losing any useful info

Updated the PR description to include the latest changes: rc/ec in error messages for better diagnostics, and CHANGELOG entry for CHECK0 removal. Title still fits — kept as is.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 4 comments.

Comment thread src/tls/cert.h
Comment thread src/crypto/openssl/verifier.cpp
Comment thread src/crypto/openssl/rsa_public_key.cpp
Comment thread src/crypto/openssl/rsa_key_pair.cpp
@maxtropets maxtropets merged commit 55e5977 into main Apr 16, 2026
19 checks passed
@maxtropets maxtropets deleted the copilot/add-check1-to-crypto-files branch April 16, 2026 12:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-long-test Run Long Test job

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants