Repository navigation
BUG: Require fiftyone_pipeline_core at the version that holds the names four packages import - #257
Draft
jwrosewell wants to merge 2 commits into
Draft
jwrosewell wants to merge 2 commits into
jwrosewell wants to merge 2 commits into
Conversation
… hold The four rows of the requirements test named MissingPropertyReason, CloudMissingPropertyService and OnPremiseMissingPropertyService in the engines and Cloud Request Engine packages. Those classes were removed before the test merged, so 8 of its checks fail on every build. The rows are now the four names a package here imports from another that the other's last release does not hold, all of them in fiftyone_pipeline_core, whose last release is 4.5.43: _contents_property, _CheckedDictWrites and, twice, ElementDataPool. The imports and the modules holding the names pass. The requirements fail, as none of the four packages requires fiftyone_pipeline_core at 4.5.44.
…es four packages import Four packages import a name from fiftyone_pipeline_core that its last release, 4.5.43, does not hold, and none required the version that does. Upgrading one of them alone left the older core in place, and the import failed. fiftyone_pipeline_engines (_contents_property), fiftyone_devicedetection_cloud (_CheckedDictWrites) and the two on-premise packages (ElementDataPool) now require fiftyone_pipeline_core>=4.5.44. The comments on three other requirements named classes that no longer exist. The versions they require are unchanged, and each comment now says only that it is the release the package is built and tested with. The core package's tox.ini takes core from the checkout before its wheel is built, as the engines package does, because the engines package it installs now requires a core that a package index may not have yet.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The unit tests of
fiftyone_pipeline_enginesfail onrcas it stands, intests/test_package_requirements.py, so every full build ofrcstops in the Pipeline's unit tests.rcat d85e95b with nothing changed but its own workflow file. Its job Ubuntu_Python_3.12 gave "8 failed, 130 passed" for the engines package.rc, 60e1b5e, the same 8 fail here (Windows, Python 3.12.10): 8 failed, 132 passed.The nightly on
mainbuildsrcas well, and will show the same failure as soon as its Configure job passes again. With the next release the test reachesmainitself.Cause
MissingPropertyReasonandOnPremiseMissingPropertyServicetofiftyone_pipeline_engines.missingproperty_service, andCloudMissingPropertyServicetofiftyone_pipeline_cloudrequestengine.clouddata.setup.pyfiles, name those three classes. So the test has failed from the commit that added it.The test's purpose still holds. A package that imports a name from another package here must require that package at the version the name came in at, or upgrading it alone leaves an older copy of the other in place and the import fails.
What the rows should be
I compared every
from <package>.<module> import <name>in the source of each published package onrcwith the module the named package holds at its latest release tag. There are 77 such imports between packages. Four name something the latest release does not hold, and all four are infiftyone_pipeline_core, whose latest release is 4.5.43 (tagfiftyone_pipeline_core/4.5.43).rcfiftyone_pipeline_engines_contents_propertyfiftyone_pipeline_core.elementdatafiftyone_pipeline_core, no minimumfiftyone_devicedetection_cloud_CheckedDictWritesfiftyone_pipeline_core.elementdatafiftyone_devicedetection_onpremiseElementDataPoolfiftyone_pipeline_core.elementdata_pool, a new module>=4.5.42fiftyone_ipintelligence_onpremiseElementDataPoolfiftyone_pipeline_core.elementdata_pool>=4.5.42So each of the four can be installed today beside a core that it cannot import from.
ci/write-package-versions.ps1gives 4.5.44 as the version this checkout releases core at.What changed
tests/test_package_requirements.pyhas the four rows above in place of the four that named removed classes, each at 4.5.44, andPACKAGE_FOLDERSnames the core package. The test's code is unchanged.fiftyone_pipeline_core>=4.5.44insetup.py, each with a comment naming what it imports. Forfiftyone_devicedetection_cloudthat is a new requirement of its own.fiftyone_pipeline_engines>=4.5.44in the Cloud Request Engine and in both on-premise packages, andfiftyone_pipeline_cloudrequestengine>=4.5.45infiftyone_devicedetection_cloud. Each comment now says "The release this package is built and tested with." The second commit's message says three where four is right.pipeline/fiftyone_pipeline_core/tox.inilists-e../fiftyone_pipeline_core/among its dependencies, as Require the version of each package that holds the names imported #231 did for the engines package in its owntox.ini. The core package's test environment installs the engines package from the checkout, which now requires a core that no package index has, so core has to come from the checkout before its wheel is built. The wheel replaces it before the tests run.No source file of any package changes.
Before you upgrade
Installing or upgrading
fiftyone_pipeline_engines,fiftyone_devicedetection_cloud,fiftyone_devicedetection_onpremiseorfiftyone_ipintelligence_onpremiseat this release bringsfiftyone_pipeline_core4.5.44 or later with it. Nothing else changes.A question for the review
The four requirements in point 3 were raised by #231 for names that are gone, and nothing imported today needs those versions. I left them where they are, because lowering a requirement is a decision of its own and a higher minimum only asks for packages that are released together anyway. If the rule is that a minimum stays only while a row of the test gives its reason, they go back to what the last release has: no minimum on
fiftyone_pipeline_enginesin the Cloud Request Engine,>=4.5.42in the on-premise packages, and no requirement on the Cloud Request Engine infiftyone_devicedetection_cloud.How it was verified
Windows, Python 3.12.10, with
COMMON_CInaming a checkout of common-ci'smainand the package versions written byci/write-package-versions.ps1(core 4.5.44, engines 4.5.44, Cloud Request Engine 4.5.45).pytest tests/test_package_requirements.pygives 4 failed, 3 passed and 8 subtests passed. The imports and the modules holding the names pass for all four rows. The four failures are the requirements, for exampledevice-detection/fiftyone_devicedetection_onpremise requires fiftyone_pipeline_core >=4.5.42, below the 4.5.44 that holds ElementDataPool.tox -e py312inpipeline/fiftyone_pipeline_engines: 132 passed, 0 failed, wherercgives 8 failed and 132 passed.tox -e py312inpipeline/fiftyone_pipeline_core: the environment is built and the tests run, giving the 11 failuresrchas in that package and 616 passed. Those 11 have another cause, fixed in #256.tox.iniline taken out is not built:ERROR: No matching distribution found for fiftyone_pipeline_core>=4.5.44 (from fiftyone-pipeline-engines).tox -e py312 --notestindevice-detection/fiftyone_devicedetection_cloudbuilds its environment withfiftyone_pipeline_cloudrequestengine>=4.5.45 fiftyone_pipeline_core>=4.5.44, both from the checkout.Invoke-Pester ci/write-package-versions.Tests.ps1: 4 passed.tox.inithat installs the engines package from the checkout already lists core from the checkout, apart from core's own, which this changes.What remains
fiftyone_pipeline_corestill fails 11 tests onrcuntil BUG: Refuse writes through a view of a finished Flow Data's Element Data #256 merges.