Repository navigation
gh-141276: Avoid compiling zipimport source in get_filename - #153799
Conversation
|
This fixes the duplicated SyntaxWarning from #141276, could someone take a look? |
itamaro
left a comment
There was a problem hiding this comment.
the fix generally looks good to me, thanks for the PR!
see inline comments regarding skipping wasted source load.
a few additional requests:
- this is an observable behavior change, which needs to be called out in the NEWS.
the semantic change - previously, py files with syntax errors in zip would raise from find_spec() and get_filename(). after this change, find_spec() and get_filename() would succeed, and only exec_module() would raise.
while this sounds like a significant behavior change, I think it's acceptable and even desirable, because it brings zipimport in line with the standard filesystem based finder/loader - but it does warrant more explicit coverage in NEWS.
this also makes it somewhat risky to backport, so I'd like @serhiy-storchaka's opinion here as well.
- I'd like to see a few more test cases covering the expected behavior, and preventing unintended future regressions, especially around the subtler corners.
a test that with a py with SyntaxError. assert that find_spec() and get_filename() succeed and return expected spec / filename, and exec_module() raises SyntaxError.
perhaps also have a test that calls get_filename() directly and asserts no warnings.
I see testBadMagic/testBadMTime already cover stale/bad-magic .pyc falling back to .py. could you add:
- valid
.pyc+.pywith a syntax error → spec picks the.pycand import succeeds - package
__init__.pywith a syntax error + a valid sibling module of the same name → origin is__init__.pyand exec raisesSyntaxError(no fallthrough to the module)
|
A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated. Once you have made the requested changes, please leave a comment on this pull request containing the phrase |
Move the compile_source=False early return above the archive read, so get_filename() no longer decompresses source it immediately discards. Also update the NEWS entry with itamaro's suggested wording and add tests covering the deferred-SyntaxError behavior: find_spec()/ get_filename() succeed and get_filename() emits no warnings for a module with a syntax error, a valid .pyc still wins over a bad sibling .py, and a package's __init__.py error doesn't fall through to a same-named sibling module.
itamaro
left a comment
There was a problem hiding this comment.
thanks for making the requested changes!
please don't forget to follow the bot instructions about commenting on the PR once you've made the requested changes (it's needed so the bot updates PR labels and makes it show up in review queues again, otherwise it's more likely to go unnoticed).
we also need to update the docs (Doc/library/zipimport.rst) to match the modified get_filename behavior (raises when "module couldn't be found", not when "module couldn't be imported".
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
I have made the requested changes; please review again. Pushed 8082336 with the NEWS wording, the docstring, the comment, the redundant |
|
Thanks for making the requested changes! @itamaro: please review the changes made to this pull request. |
|
thanks, it looks good to me! I'll merge main since it's been a while, and merge if CI passes. cc @serhiy-storchaka @hugovk what do you think about backporting this? (see previous comment) |
Documentation build overview
|
Summary
Root cause
zipimporter.get_filename()called_get_module_code(), which compiled source while import machinery was creating the module spec. Module execution then compiled the same source again.Tests
./python.exe -m test test_zipimport -m test_syntax_warning -m testBadMagic -m testBadMTime./python.exe -m test test_zipimportRefs #141276
Duplicate-work check
No open or merged pull request referencing #141276 was found before submission.
AI assistance
OpenAI Codex assisted with investigation, implementation, testing, and review.