Skip to content

tpm2: Fix memory leak of private exponent D in ComputePrivateExponentD - #655

Merged
stefanberger merged 1 commit into
masterfrom
master.next
Nov 3, 2026
Merged

stefanberger merged 1 commit into
masterfrom
master.next

Conversation

@stefanberger

@stefanberger stefanberger commented Nov 3, 2026 •

Copy link
Copy Markdown
Owner

When the Carmichael-computed private exponent D is too small, the code falls back to the Euler computation, which assigns a newly allocated BIGNUM to *D and thereby leaks the one allocated by the Carmichael computation. Free the first D (clearing it since it is secret) and reset the pointer before falling back.

Summary by CodeRabbit

  • Bug Fixes
    • Improved reliability of cryptographic key calculations by ensuring an earlier intermediate result is cleared before an alternate calculation is used. This helps prevent an unsuitable result from affecting the final calculation.

When the Carmichael-computed private exponent D is too small, the code
falls back to the Euler computation, which assigns a newly allocated
BIGNUM to *D and thereby leaks the one allocated by the Carmichael
computation. Free the first D (clearing it since it is secret) and reset
the pointer before falling back.

Signed-off-by: Stefan Berger <stefanb@linux.ibm.com>
@coderabbitai

coderabbitai Bot commented Nov 3, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review.

Walkthrough

When the Carmichael-derived D has at most half the bit length of N, ComputePrivateExponentD now clears D and sets it to NULL before computing the Euler fallback.

Changes: Private exponent calculation

Layer / File(s) Summary
Clear D before Euler fallback
src/tpm2/TPMCmd/tpm/cryptolibs/Ossl/Helpers.c
The fallback branch clears the existing Carmichael-derived D and sets it to NULL before computing the Euler result.

Estimated code review effort: 2 (Simple) | ~5 minutes

Change: Bug fix

Merge Risk: Minimal · up to cd7ba

The targeted private-exponent cleanup is safe and introduces no identified merge-blocking risk.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentat… Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the specific fix: a memory leak of the private exponent D in ComputePrivateExponentD. The raw summary and objectives both describe clearing, freeing, and resetting the Carmichael-deriv…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names the specific fix: a memory leak of the private exponent D in ComputePrivateExponentD. The raw summary and objectives both describe clearing, freeing, and resetting the Carmichael-derived D before the Euler fallback. The title is specific and matches the main change, and it is 70 characters long, within a typical title length.

Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff with cyclomatic complexity of at least 2. Documentation-only edits and deletions within surviving functions are included; unchanged diff context and fully deleted functions are excluded. Analyzed 1 functions across 1 files.

Resolution

Write docstrings for the functions missing them to satisfy the coverage threshold.

Finishing Touches
Generate docstrings
  • Commit to this branch
  • Create a new PR
Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Some tools did not complete. Review the errors below.

ast-grep (0.51.1)

ast-grep could not parse rule config: /ast-grep-rules/ast-grep-essentials/rules/rust/security/weak-hash-md5-rust.yml

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

CodeRabbit Verify

  • CodeRabbit Verify could not start. Follow the reply for details, then check again.
Recent review info
Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 9b1aa831-4e39-4455-be98-a603c026cb34
Commits

Reviewing files that changed from the base of the PR and between ff2be88 and cd7ba9c.

Files selected for processing: 1
  • src/tpm2/TPMCmd/tpm/cryptolibs/Ossl/Helpers.c

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

@coderabbitai

coderabbitai Bot commented Nov 3, 2026

Copy link
Copy Markdown

CodeRabbit Verify cannot start because this repository is not enabled for PR Verify. Enable PR testing in Verify settings.

CodeRabbit Verify can be triggered again using the @coderabbitai verify command.

@stefanberger
stefanberger merged commit 6ebf9d3 into master Nov 3, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant