Skip to content

gh-141276: Avoid compiling zipimport source in get_filename - #153799

Merged
itamaro merged 5 commits into
python:mainfrom
harjothkhara:codex/gh-141276-zipimport
Oct 10, 2026
Merged

itamaro merged 5 commits into
python:mainfrom
harjothkhara:codex/gh-141276-zipimport

Conversation

@harjothkhara

@harjothkhara harjothkhara commented Jul 16, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • avoid compiling zipimport source while resolving a module filename
  • preserve bytecode validation and source fallback behavior
  • add a regression test and a NEWS entry

Root cause

zipimporter.get_filename() called _get_module_code(), which compiled source while import machinery was creating the module spec. Module execution then compiled the same source again.

Tests

  • ./python.exe -m test test_zipimport -m test_syntax_warning -m testBadMagic -m testBadMTime
  • ./python.exe -m test test_zipimport

Refs #141276

Duplicate-work check

No open or merged pull request referencing #141276 was found before submission.

AI assistance

OpenAI Codex assisted with investigation, implementation, testing, and review.

@brettcannon
brettcannon removed their request for review July 17, 2026 19:12
@harjothkhara

harjothkhara commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor Author

This fixes the duplicated SyntaxWarning from #141276, could someone take a look?

@itamaro itamaro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the fix generally looks good to me, thanks for the PR!

see inline comments regarding skipping wasted source load.

a few additional requests:

  1. this is an observable behavior change, which needs to be called out in the NEWS.

the semantic change - previously, py files with syntax errors in zip would raise from find_spec() and get_filename(). after this change, find_spec() and get_filename() would succeed, and only exec_module() would raise.

while this sounds like a significant behavior change, I think it's acceptable and even desirable, because it brings zipimport in line with the standard filesystem based finder/loader - but it does warrant more explicit coverage in NEWS.
this also makes it somewhat risky to backport, so I'd like @serhiy-storchaka's opinion here as well.

  1. I'd like to see a few more test cases covering the expected behavior, and preventing unintended future regressions, especially around the subtler corners.

a test that with a py with SyntaxError. assert that find_spec() and get_filename() succeed and return expected spec / filename, and exec_module() raises SyntaxError.

perhaps also have a test that calls get_filename() directly and asserts no warnings.

I see testBadMagic/testBadMTime already cover stale/bad-magic .pyc falling back to .py. could you add:

  • valid .pyc + .py with a syntax error → spec picks the .pyc and import succeeds
  • package __init__.py with a syntax error + a valid sibling module of the same name → origin is __init__.py and exec raises SyntaxError (no fallthrough to the module)

Comment thread Lib/zipimport.py
Comment thread Lib/zipimport.py Outdated
Comment thread Misc/NEWS.d/next/Library/2026-07-15-21-45-33.gh-issue-141276.xH7qP2.rst Outdated
@bedevere-app

bedevere-app Bot commented Oct 3, 2026

Copy link
Copy Markdown

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Move the compile_source=False early return above the archive read, so
get_filename() no longer decompresses source it immediately discards.
Also update the NEWS entry with itamaro's suggested wording and add
tests covering the deferred-SyntaxError behavior: find_spec()/
get_filename() succeed and get_filename() emits no warnings for a
module with a syntax error, a valid .pyc still wins over a bad
sibling .py, and a package's __init__.py error doesn't fall through
to a same-named sibling module.

@itamaro itamaro left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for making the requested changes!

please don't forget to follow the bot instructions about commenting on the PR once you've made the requested changes (it's needed so the bot updates PR labels and makes it show up in review queues again, otherwise it's more likely to go unnoticed).

we also need to update the docs (Doc/library/zipimport.rst) to match the modified get_filename behavior (raises when "module couldn't be found", not when "module couldn't be imported".

Comment thread Misc/NEWS.d/next/Library/2026-07-15-21-45-33.gh-issue-141276.xH7qP2.rst Outdated
Comment thread Lib/zipimport.py Outdated
Comment thread Lib/zipimport.py Outdated
Comment thread Lib/zipimport.py Outdated
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@harjothkhara

Copy link
Copy Markdown
Contributor Author

I have made the requested changes; please review again.

Pushed 8082336 with the NEWS wording, the docstring, the comment, the redundant modpath line and the Doc/library/zipimport.rst update.

@bedevere-app

bedevere-app Bot commented Oct 10, 2026

Copy link
Copy Markdown

Thanks for making the requested changes!

@itamaro: please review the changes made to this pull request.

@itamaro

itamaro commented Oct 10, 2026

Copy link
Copy Markdown
Contributor

thanks, it looks good to me! I'll merge main since it's been a while, and merge if CI passes.

cc @serhiy-storchaka @hugovk what do you think about backporting this? (see previous comment)

@read-the-docs-community

Copy link
Copy Markdown

Documentation build overview

📚 cpython-previews | 🛠️ Build #35063365 | 📁 Comparing cb21019 against main (5564790)

  🔍 Preview build  

2 files changed
± library/zipimport.html
± whatsnew/changelog.html

@itamaro
itamaro merged commit 230917f into python:main Oct 10, 2026
55 of 56 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants