Repository navigation
Reuse Element Data through a bounded pool for each Flow Element - #248
Merged
Merged
Conversation
Each package that defines a Flow Element gains a test that processes every Flow Element the package and its examples define, then writes to its Element Data and expects an error. It also checks that it covers every Flow Element class the package's own modules define. The pipeline core test also checks that a finished Flow Data refuses Evidence, Element Data and errors, that a second process raises even when process exceptions are suppressed, and that Flow Elements sharing one Element Data write it while each processes.
FlowData raises once processing has finished when Evidence, Element Data or an error is added, and a second process raises without adding an error. ElementData records when the Flow Element that writes it has finished, and then setting a public attribute, or a write method such as the derived property data's set_value, raises. A Flow Element that shares the instance opens it again while it processes. Names that start with an underscore hold the data's own state, such as values kept once read, and closing the Flow Data is not a write. The checks read a field set when processing ends and take no lock.
The pipeline core test counts the Element Data the Sequence, JSON Bundler, JavaScript Builder and Set Header elements make over 200 requests, each closing its Flow Data, and expects one each rather than one for each request. With more Flow Data open at once than a pool's bound, the instances made stay at the bound and the pool keeps no more than it. Threads processing at once make no more instances than there are threads. A reused instance has none of the last request's values, is open for writing while its Flow Element processes and refuses writes once it has finished. Closing a Flow Data twice gives an instance back once, and the instances of a Flow Data that is not closed go to the garbage collector. The engines test checks that an instance a cache holds is never reused. The Device Detection and IP Intelligence on-premise tests count the results data the engines make over 200 requests and expect one, and check that a reused instance gives the new request's results.
ElementDataPool, in the pipeline core, keeps up to twice the number of processors of the Element Data one Flow Element makes, behind a lock of its own, as requests on several threads share it. take() hands out an instance an earlier request finished with, after ElementData._clear() has dropped its values and opened it for writing, or None, when the element makes a new one. FlowData.close() gives each Element Data back to the pool of the Flow Element that made it, after closing it. Data from an element with a cache is never given back, as the cache may share it with other requests, and a second close does nothing. A Flow Data that is not closed leaves its instances to the garbage collector. The Device Detection and IP Intelligence on-premise engines and the Sequence, JSON Bundler, JavaScript Builder and Set Header elements take their Element Data from a pool. The FlowData.close and ElementData docstrings say that a reference to Element Data kept after its Flow Data is closed is not supported, and reads are not checked for it.
The contents of ElementDataDictionary, AspectDataDictionary and CloudData, and a hardware profile, which is itself a dict, can still be written once the Flow Element that provides them has finished. Pipeline core now checks each dict write method and that the contents stay a dict. Each package's check over every Flow Element also writes an item, and the Device Detection cloud check writes to a hardware profile in each way.
FlowData.close() read _element_data_pool from the FlowElement, so a user's element with an attribute of that name holding something else made close() fail. FlowElement.get_element_data_pool() returns that attribute only when it is an ElementDataPool, and None otherwise, and close() asks for the pool through it. The tests for this are in the commit before this one. That commit's message describes another change by mistake. Its test file is the one this message describes.
The resource cleanup page of the specification says code comments and documentation must say that keeping a reference to Element Data after its Flow Data is disposed is not supported. The comments said so, and now the Pipeline Core readme does too. Two comments say why the code is as it is. The pool's cache check is on the element, not the instance, and close() gives data back only once its close has returned, so data written by a lazy task goes back only after that task has finished.
rc now holds the read only Flow Data and finished Element Data from #247, whose earlier commits this branch also carried, and the linked list of Element Data from #250. The merge result is rc with this branch's pool commits replayed on it, and nothing else from the branch's copies of #247. In flowdata.py, _give_back sits after rc's _GIVEN table. close() keeps rc's call of the close method, which opens the Element Data itself, in place of the branch's own _set_finished calls around it, and then gives the instance back to its pool. The second close guard and the once only give back for data held under two keys are kept, and they read the Element Data through rc's data view. The core readme keeps rc's web integration section and adds the section saying Element Data is reused.
With the Element Data in a linked list, close() reads it through the data view. One instance added under a second key is in the list twice, and close() still gives it back to its pool once.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The problem
Every request makes new Element Data for each Flow Element that adds some, and the garbage collector frees it later. The specification now asks implementations that can to reuse Element Data through an object pool, and requires code comments and documentation to say that a reference to Element Data kept after its Flow Data is closed is not supported.
This draft is stacked on the draft that makes writes to a finished Flow Data fail, branch
remote/finished-writes-failat eae137e, because a pooled instance has to come back open for writing, which needs that draft's record of when a Flow Element has finished. It retargets torconce that draft merges.The specification
From resource-cleanup.md, Reusing Element Data, in specifications#190 at a90ba516.
From resource-cleanup.md, Flow Data.
In Python a Flow Data is disposed by
FlowData.close(), which awithstatement calls.The change
At the head, 5bdd9b3.
ElementDataPool, new in the pipeline core, keeps up toDEFAULT_MAXIMUM_INSTANCES, twice the number of processors, of the Element Data one Flow Element makes.take()hands out an instance an earlier request has finished with, after its_clear()has dropped its values and opened it for writing, orNonewhen the pool holds none, and the Flow Element then makes a new one.give_back()keeps an instance unless the pool is full or the Flow Element that made it has a cache. Requests on several threads share the pool, so it takes athreading.Lockof its own. That lock guards state shared between requests, and no Flow Data or Element Data is locked.ElementData._clear()opens an instance for writing.ElementDataDictionaryalso drops its contents, with a new dictionary so that one returned earlier byas_dictionary()is unchanged. The Device DetectionSwigDatadrops its native results, and the IP IntelligenceSwigDataits native results and the addresses it echoes.FlowData.close()closes each Element Data as before, then gives it back to the pool of the Flow Element that made it, if that element keeps one as_element_data_pool. An instance held under two keys goes back once, and a secondclose()does nothing, so one instance never reaches two requests. A Flow Data that is not closed gives nothing back, and its instances go to the garbage collector as before.FlowData.closeandElementDatadocstrings say that a reference to Element Data kept after its Flow Data is closed is not supported, because another request may then change its values, and that reads are not checked for it.An instance is cleared when it is taken, not when it goes back, so its values stay readable after
close()until another request takes it. None of the elements adopted here writes its Element Data from a lazy task, soclose()gives each back at once.No existing public name changes. The new public names are
ElementDataPoolandDEFAULT_MAXIMUM_INSTANCESinfiftyone_pipeline_core.elementdata_pool, and the rest are private.Flow Elements not adopted
CloudEngine(Device Detection, IP Intelligence, country translation and hardware profile). The call to the cloud service is a lazy property that thread-safety.md lines 164 to 186 puts in a lazy task, and an instance goes back only once any lazy task writing it has finished, so these follow the lazy task pattern into this repository.derivedthat the first of them makes, so they are left for a change of their own._create_element_data, so each kind needs its own_clear()first.Commits
remote/finished-writes-fail.Released tests
The tests of the last release,
mainat 612aa28, ran unaltered against this branch for each package whose code or tests it touches and for the two examples packages that use the adopted engines and elements. The names of the failing tests are the same as onrcat 2373d14 and onremote/finished-writes-failat eae137e, package by package, with none new and none fixed.The released tests that exercise what this touches, on this branch.
test_close.pytest_core.pytest_javascriptbuilder*.py(5 files)test_jsonbundler*.py(6 files)test_setheader.pytest_web.py,test_web_client_side.py,test_web_integration_elements.pytest_devicedetection.pywithstatement, andtest_flowdata_close_releases_results, which reads after closingtest_swig.py,test_properties.py,test_read_by_index.py,test_swig_data_index.pySwigDatatest_ipintelligence.pytest_flowdata_close_releases_resultstest_read_by_index.py,test_swig_data_decimal_places.py,test_swig_data_index.pySwigDataThe failures that are the same on
rcaretest_setheader.pytest_get_response_header_name_1_SetHeaderbrowserAccept_CH, five intest_devicedetection.py(test_excluded_property,test_missing_property_service_not_found_anywhere,test_missing_property_service_not_found_in_current,test_validate_data_false,test_validate_data_true),test_update_matched_useragent.pytest_turned_off_leaves_user_agents_empty,test_ipintelligence.pytest_missing_licence_key, the 31 in fiftyone_pipeline_engines (test_datafile_update_http.py,test_datafile_update_service.py,test_engines.py,test_licence_key_masking.py), and the cloud examples, run here with no resource key.The red and the green
Both ran through a throwaway build of each commit, without macOS, on Ubuntu and Windows with Python 3.10 to 3.14.
Both throwaway branches are deleted.
Local counts
On Windows with Python 3.14.5, with the free Lite and ASN data files.
remote/finished-writes-failand pass on this branch. In pipeline core, 9 tests count the Element Data the four core elements make over 200 requests (one each), the instances made with more Flow Data open at once than a pool's bound and from four threads at once, and check that a reused instance has none of the last request's values, is open for writing and then refuses writes, is given back once when closed twice, and that the instances of Flow Data that is not closed go to the garbage collector. In fiftyone_pipeline_engines, 2 tests check that an instance a cache holds is never reused and that one from an engine without a cache is. In each on-premise package, 2 tests count oneSwigDataover 200 requests and check that a reused instance gives the new request's results.remote/finished-writes-failthe same suites gave 571, 127, 104 and 79 passed, the difference being the new tests.What remains
rconceremote/finished-writes-failmerges.ElementDataPoolfromfiftyone_pipeline_coreand still ask forfiftyone_pipeline_core>=4.5.42. Their minimum moves to the core release that carries the pool when they are released, so that an on-premise package is never installed beside a core without it.close()gives an instance back only once any lazy task writing it has finished.Before you upgrade
close()or at the end of itswithblock, a later request reuses the Element Data it held from the Device Detection and IP Intelligence on-premise engines and from the Sequence, JSON Bundler, JavaScript Builder and Set Header elements. A reference to that Element Data kept after closing can then show another request's values. Closing a Flow Data a second time now does nothing.AspectPropertyValueor the JSON from the JSON Bundler, are separate objects and stay as they were. Code that never closes its Flow Data is unaffected, as its Element Data is left to the garbage collector.