Repository navigation
Cite US and UK spec sources by symbol or at a commit, never by a drifting line - #1407
Merged
Merged
Conversation
…ting line Rebuild #1146 on current main without its NZ files; #1282 landed the NZ half with its own stricter grammar. Add the shared validator (test_support/microcosm_build/spec_source_citations.py) and an engine-free guard over every country package. It refuses a line citation of a file in this repository, pinned or not, and a line citation of any other source unless its string names the commit or the resource pins the file in a path/commit mapping. Every dotted microcosm.<shard> name must resolve. US: port #1146's known-gap and source-stage rewrites that still apply (main dropped the employer-premium gap), and name archived commit 42ed5d45 in the ESI stage note added since. UK: the parity register's 13 citations name its incumbent commit a43e8c5, each re-read there (hmrc_cgt 195 -> 196, hmrc_spi 230 -> 229, frs_only.py -> services.py 137-148); the population contract's citations name registry ref 12a1e028 (30 ported strings, 9 rewritten since #1146 re-read there; compute_esa is at income.py 98-102 at that ref). Re-pin the source_stages.json raw hash and regenerate us/spec/sources.yaml. The digests derived from it follow in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The source-stage citation edits move the US spec_sha256 (c677d50b -> ca232744) and documentation_sha256 (801a99da -> 3cb60930), and three inventory digests: source_manifest (87b1e124 -> ecf9ca2b), late_resource_semantics (531b48cd -> e8797144) and full_checkpoint (030c8422 -> 189d4e28). Regenerated us-f0-coverage.json with tools/spec_engine_coverage.py: 41/41 inventory checks, and exactly ten leaves move (the five above, their observed copies and the pool-code observed digest). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bind each line citation to the nearest inline commit in its own clause, else the nearest one before it, and drop that fallback when the clause names a moving ref. Refuse commit-like tokens in no pin spelling, widen the locator grammar with a backstop for unrecognised spellings, strip this-repository prefixes and count only anchored path tails, make a path pin cover only citations whose components end its path, and stop reading truncated hashes as pins. Name the archived commit before the US PUF note's first citations and give the ESI origin its own commit, then re-pin every digest built on us/source_stages.json. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The misspelled-pin property placed its "between" token after the clause break, inside the citation's own clause, so dropping the gap rule left it passing. Add a placement in the binding commit's clause. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Accept quotes before a colon or # locator's number, so x.py:"12-19" is a citation. Count raw.githubusercontent.com URLs at a branch, refs/heads/ branches and quoted branch names as moving refs, so an earlier commit no longer pins them. Stop reading a prose pin (commit, at, @) whose sha runs on into a path: in "at a1b2c3d/foo.py:12" the hex is a directory. Add a property for each, plus the review's exact inputs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The in-repository rule removed a GitHub URL's ref as one path segment, so .../microcosm/blob/refs/heads/main/tools/x.py#L12 (and any slash-named branch) left heads/main/tools/x.py unresolved and passed when pinned. Try the path after each segment instead, and add a property over refs of one to four segments plus the round-3 review's exact input. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 4 of the PR 1407 review found two valid GitHub URLs that escaped the guard: `%74ools/x.py` lost the repository prefix during extraction, and a `?plain=1&review=...` query longer than the backstop's window hid the `#L12` locator. Each URL in a scanned string is now read the way GitHub reads it: scheme and host lower-cased, a user or port dropped, `%` escapes of unreserved characters and `/` decoded, dot segments resolved, and the query dropped. GitHub reads `%23` and `%3F` as part of a file name, so those stay escaped. The resolver also tried every split point of a URL's ref, so an 8,000-segment ref took 115 s and 1.09 GB. It now tries only the paths no longer than the deepest path form in the checkout. The new reading changes 69 live strings (queries on data-source URLs), none of which carries a citation: classification is unchanged across all 200 resources and 283 citations. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The new URL property could escape the `2` of a `%2E` detour, writing `%%32E`, which names a different path from the one it expects refused. The mutant harness's unmutated baseline caught it (raw URL, ref 000000a, detour `x/%2E%2e/`). Characters already inside an escape now stay as written; the helper is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 5's review found that a URL of this repository with an escaped host (https://%67ithub.com/...) or a network-path reference with a port or a user (//github.com:443/..., //user@github.com/.../%74ools/...) escaped the in-repository rule. urlsplit kept the host's escapes, and the URL pattern read a scheme-less GitHub URL only when a slash followed its host. The reader now parses the authority as the URL Standard does. Before the query, \ is /. Any number of slashes may follow https: or start a network-path reference. The user (up to the last @) and the port are dropped. The host is percent-decoded, mapped to ASCII as UTS46 maps it (NFKC case folding with Python's Unicode 16, plus Unicode 17's U+A7F1) and lower-cased. A trailing dot is dropped, since GitHub redirects github.com. to github.com. A URL the standard refuses is read as written. A scheme-less GitHub URL may now carry a user and a port. Checked against bun's WHATWG URL with every code point escaped into a host: the two read the same wherever the standard maps it into plain ASCII. Of 26,350 generated URLs, none is read only by the standard and none is read differently. The live packages' 4,482,233 strings read exactly as before. Tests: - the review's three inputs and nine more spellings, plus four authorities the standard refuses; - a table of 20 readings recorded from the standard; - the URL property now writes 12 leads, 7 authorities, full-width and escaped host characters, trailing dots and backslash separators; - a deterministic example of a clause break inside a citation (M27). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ear time Review round 6 found that a URL of this repository with a quote in its query (`.../tools/ci_test_plan.py?value='xxxx'#L12`) escaped the in-repository rule: the URL token ended at the quote, so its `#L12` was read as prose. A browser reads a quote or a bracket inside a URL as part of it (bun's URL), and GitHub serves each such URL. A URL now runs to whitespace and does not end in a quote, a bracket or sentence punctuation. The guard's scans are now linear in the string: - a GitHub host written without a scheme is matched where it starts, with any user before it left as prose, instead of scanning for a user from every character of a run; - a source-name pattern is tried only where a match can start (the start of a run of path characters, `http(s):`, or the previous match's end), which finds the same matches as `finditer`; - each string's clause breaks, inline commits and commit-like tokens are found once, and each citation's clause, nearest commits and tokens are found by bisection; the moving-ref check is cached per clause; - the backstop checks whether a near match ends inside a citation by bisection. Tests: the reviewer's input with each quote and bracket in a query or a dot segment; the URL property writes quotes and brackets in queries and dot detours, and its full-width, escape and backslash indices always land; a timing test of eight hostile strings (0.16 s; the round-6 helper takes over 90 s on five of them); a differential of the run-start scan against `finditer`, and of the bindings, misspelled pins and backstop against the whole-string scans they replace. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The mutant that stops reading U+3002 as "." in a host survived round 7: no test wrote a host with an ideographic full stop. The URL property now writes a dot at a full-width index as any of the full-width, ideographic and half-width ideographic full stops, the readings table has a raw and an escaped ideographic stop, and the trailing-dot test drops each stop. bun 1.3's URL reads all three as "." in a host. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… time A URL's query no longer hides a Markdown link or an autolink written after it: a string is also read with each URL ending at a quote, a parenthesis or an angle bracket, as Markdown and prose delimit one, and every rule applies to both readings. Line-number lists are read with possessive whitespace and an unambiguous separator, a reverse locator is tried once per list rather than once per number, and a host label longer than DNS allows is not Punycode-encoded, so no locator scan backtracks exponentially or quadratically. The patterns as they were are kept in the test file as the reference a differential property checks the new ones against. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ead Markdown and HTML escapes and URL line breaks Round 8's review found two citations of this repository that no rule saw. `L1-L2-3 of tools/ci_test_plan.py`: a reverse locator was tried only from a list's head. It read `L1-L2`, failed, and the scan moved past the list, but from `L2` the locator reads `L2-3` and succeeds. When a reverse locator fails from a list's head, `_finditer` now tries it again from the list's last number, if that is an `L` number, and only then moves past the list. From any other number it reads the rest of the list as the head did. Each number is read in at most two lists, so the scan stays linear. `[a](https://github.com/PolicyEngine/microcosm/blob/<sha>/tools/ci_test_plan\.py#L12)`: CommonMark decodes `\.` in a link destination, and character references likewise (`#L12`, `.py`). A string holding a backslash before ASCII punctuation or a character reference is now read again with them decoded, in one pass. The URL Standard also deletes tabs and line breaks from a URL, so each string is also read with them deleted, as written and as decoded. Every rule checks every reading, so a new reading can only refuse more. Tests: the review's inputs, three more chains, 15 escaped Markdown and HTML shapes and 4 broken hrefs; a property that escapes a line link of this repository in six Markdown and HTML settings; chain pieces and examples in both differentials; and timing strings for the round-8 mutant survivors M80 and M81 and for the new code. Three tests that asserted the exact citation list now filter to the first reading. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ugh the whitespace a link holds Round 9's review found an href this guard did not read as a browser does: `<a href=".../tools\ci_test_plan.py#L12">`. HTML decodes the reference and leaves the backslash, which the URL Standard reads as `/`; Markdown's one pass took `\&` as an escaped `&`, so no reading decoded the reference. A string holding a `&` or a backslash is now decoded as Markdown decodes it and, apart, as HTML does, each once as text and once as a URL holds what it decodes to (whitespace percent-encoded, a tab or line break deleted). References are decoded as the HTML standard decodes them: a reference to a control is the control (Python's `html.unescape` drops it, so `..` read as `..`), one to a C1 control is read as windows-1252 reads it, one to 0 or a surrogate is U+FFFD, and in a URL a legacy name without `;` before a letter, a digit or `=` stays as written. Beyond the review: whitespace in a segment that a later `..` removes. A quoted attribute value and a Markdown destination in angle brackets hold any whitespace, so each that starts a URL is now read as the browser takes it, its ends stripped, its tabs and line breaks deleted and its spaces percent-encoded. A Unicode space such as U+00A0 ends no URL in the URL Standard, so where a string holds one its URLs are also read through it. And an in-repository path's inner `..` segments are now resolved (`tools/x/../ci_test_plan.py#L12 at <sha>` is refused); a path that climbs out above where it starts names nothing here. Every rule checks every reading, so a new reading can only refuse more. Over all 4,482,233 live strings, every rule's output is unchanged and no string is read a new way. Tests: 25 counterexamples, each confirmed by parse5 7.3, markdown-it-py 4.2 and bun 1.3's URL; exact readings for the new decoders; inner and leading `..` paths; three plain strings the new readings must not refuse; the markup property now also inserts a whitespace detour before a `..` and turns slashes into backslashes in HTML; and timing strings for each new pattern. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…it do A fuzz of round 10's generation against markdown-it-py 4.2, parse5 7.3 and bun 1.3 found five links no reading refused. HTML decodes an attribute's references before the URL Standard strips its C0 controls and spaces, so `href=" https://..."` and `href="https://..."` are links; the enclosed reading held the decoded space as `%20` first, and the value no longer started a URL. markdown-it strips whitespace from a destination's ends before following it (`str.strip`, in mdurl), so `[a](< https://... >)`, a destination padded with U+3000 or U+00A0, and `[a](< https://...>)` are links too; the angle reading did not strip at all. Each enclosed value is now decoded without holding, then stripped as its renderer strips it, then held. Over 200,000 generated links, 113,469 render to a line link of this repository and each is refused as written. The five shapes are in the counterexample test. The round-10 timing strings get 2 s of their own: all grow linearly, but under load they pushed the shared budget over. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s query's whitespace Round 10's review found four links of this repository that no rule refused, each confirmed with markdown-it-py 4.2, parse5 7.3 and bun 1.3: an angle destination after the space CommonMark allows before its `<`, an attribute whose opening quote a code span's `="` had consumed, a backslash that Markdown writes as `%5C` and a reading took as `/`, and an ordinary destination padded with ` `. In each, a `..` removes a segment that holds whitespace or a backslash, and the renderer removes another segment than a reading does. Every line citation in a string that holds a dot segment is now refused, pinned or not (`dot_segment_line_citations`): `..` as a whole segment beside `/` or `\`, either dot perhaps `%2e`, in any source a reading starts from. Without one, whitespace or a backslash stays in the segment a renderer follows, and a test checks that no source file here has either in its path. No live string holds a dot segment. GitHub ignores a query, so whitespace in one ended the URL wherever no enclosure held it, and the line link was refused only where its `#L` fell within the backstop's 24 characters. Where a string holds a `?` and a `#`, its URLs are now also read through their query's whitespace, ending the query at an angle bracket, a `"` or a `'`. An angle destination is also found after spaces, tabs and one line break. The counterexample test runs the four findings and their variants (11 inputs) and 14 query shapes, each rendered by the three oracles. Two properties cover the rule and the query reading, and a test checks the premise against this tree's file names. The review's 679 misses are now refused, its 27,625-input matrix is unchanged, and the 288 over-refusal probes without a dot segment gain no refusal. Each mutant that only random generation killed at `c36671e6`'s tests (M12 survived round 10's run that way) now has an explicit example. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…egments and paths that hold = Round 11's review found two shapes of link to a line of this repository that no rule refused. markdown-it-py 4.2, parse5 7.3 and bun 1.3 render each as a link to line 12 of `tools/ci_test_plan.py` on github.com. In the first, the ref holds punctuation at which a reading starts a name (`blob/feature+foo/...`, `release!1`, `v1,2;3=4`, `a=b`), so the citation a reading finds (`foo/tools/ci_test_plan.py`) names no file here. In the second, a code span's `="` has taken an attribute's opening quote, and whitespace in the URL's user (`https://a b@%67ithub.com/...`) ends every reading's URL before its host. The review also named `%2f` beside `..`, which GitHub reads as a dot segment. A string is now refused where any reading's source holds this repository's GitHub path, `PolicyEngine/microcosm/`, with a line fragment at the first `#` after it (`line_links_of_this_repository`), whatever file the link names. The path is read as GitHub reads one: escapes decoded, a backslash as `/`, a run of slashes as one, `.` segments removed, and the owner and name in any case. A renderer's link to a line here names that path. Between a source and its URL lie the decodings the readings already apply, these normalizations, percent-encoding of characters that are neither in the path nor a `#`, and dot segments, which the dot-segment rule refuses. Relative links and redirects are out of scope. Prose that holds such a path and fragment without linking is refused too, by intent. A `..` beside `%2f` is now a dot segment. A path here can hold an `=` (`docs/evidence/margins/period=2025-01.json`), so the run of path characters before a cited name is also tried as an in-repository path, and a reverse locator reads a name after an `=` (`lines 3 of a/b=c.py`). The counterexample test runs 22 inputs: all render as links to line 12 here, no rule at 2822aff refused any, and the new rule refuses all 22. A property draws refs with punctuation, five spellings of github.com's authority, five of this repository's path and eight enclosures, and checks that a link here is refused and its twin in another repository is not. Rendered by the oracles, 2,880 inputs from that space and 2,000 random draws agree: every input here is refused, and none elsewhere. Across 4,482,233 live strings, the earlier rules' results are unchanged, the new rule refuses none, and no string holds a dot segment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A dot segment can form this repository's GitHub path in a link to a file whose name no reading takes for a source: `https://github.com/PolicyEngine/x/../microcosm/blob/<sha>/LICENSE#L3`. markdown-it-py 4.2, parse5 7.3 and bun 1.3 render it as a link to line 3 of LICENSE here; so do `x/%2E%2e/`, `//x/../../`, `a%2fb/../`, `x\..\` and `..` in its place, and the same with this tree's Dockerfile and SHA256SUMS. No reading finds a citation, so the dot-segment rule never sees one, and the repository-link rule read the path without resolving dot segments. Of 3,960 inputs (4 files x 22 detours x 5 enclosures x 3 authorities x 3 queries), 1,920 render as a link to a line here, and at 8a62c93 1,242 of those passed every rule. The repository-link rule now reads each source as a browser and then GitHub read a URL's path: a backslash as `/` and the dot segments before each `?` or `#` resolved, either dot perhaps `%2e` and an empty segment counted; then escapes decoded and a run of slashes read as one. This single reading replaces the earlier one, whose refusals it covers. The earlier reading also refused links that a `..` sends out of this repository (`PolicyEngine/microcosm/../other/...`); this one does not. A browser reads `%2f` as no separator. GitHub decodes none in the owner or the name (404 for `x%2f..%2fmicrocosm`, measured with curl --path-as-is), so decoding one after resolving can only add a refusal. GitHub also returns 404 for `%252e%252e` and `%5c` there, and 200 for `x/%2e%2e/` and `x/.%2e/`, which a browser has already resolved. All 1,920 here-links in the matrix are refused now, and no other-repository detour is. Rendered by the same oracles, the property's space (11,200 inputs, expanded below) and 2,000 random draws agree: every input naming this repository is refused, and none naming another. Tests: `test_a_line_link_of_this_repository_is_refused_where_a_dot_segment_forms_its_path` (8 inputs refused; 5 that link no line here, such as `x/../../microcosm` and `x/.../microcosm`, not refused). The property draws three more spellings of this repository's path through a dot segment, two detours that leave it, the three files, and a query of `..` segments, with two explicit examples. The timing test gains long runs of dot segments, escaped dots, backslashes and segments after a `?`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The round-12 review (GPT-6.1 Sol, at 8a62c93) found the gap that 4dc0734 closes, with its own shape: a code span's `="` takes the attribute's opening quote, so every reading's URL ends at the space in `https://a b@%67ithub.com/PolicyEngine/x/../microcosm/blob/<sha>/tools/ci_test_plan.py?<30 letters>=1#L12`. markdown-it-py 4.2, parse5 7.3 and bun 1.3 render it as a link to line 12 of `tools/ci_test_plan.py`. No rule at 8a62c93 refused it, and rule 7 refuses it at 4dc0734. The dot-segment test now holds it, as the review asked. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n does A fuzz of rule 7 against the renderers found links it missed at 4dc0734, all Markdown destinations holding a backslash. markdown-it-py 4.2 percent-encodes a backslash (`%5C`), so in `[a](https://github.com/PolicyEngine/x\y/../microcosm/blob/<sha>/LICENSE#L3)` the `..` removes the segment `x%5Cy`, and the link is to line 3 of LICENSE here; GitHub serves it. In an HTML `href` a browser reads the same backslash as `/`, and the URL links `PolicyEngine/x/microcosm`. Rule 7 read every backslash as `/`, so it missed the Markdown link. Of 40,000 generated inputs (detours through many spellings of `.`, `..`, `/`, `\`, `?` and `#`, nine enclosures, six authorities), 5,590 render to a line link here, and rule 7 at 4dc0734 missed 58, each a Markdown link with a backslash. Rule 7 now resolves each source's dot segments twice: with a backslash read as `/`, as a browser reads it in HTML, and with it kept, as a Markdown renderer escapes it. GitHub reads `%5C` as no separator (`PolicyEngine/x%5c..%5cmicrocosm` returns 404). All 5,590 are refused now, and the earlier fuzz's 385 too. The round-13 review (GPT-6.1 Sol, at 5578c82) found the same class independently, with the inline shape. The dot-segment test adds four Markdown links: an inline link, a pointy destination, a reference definition and an autolink, each refused by no rule at 4dc0734. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The round-13 review noted that rule 7's property drew no path through a segment holding a backslash, the shape its finding used. The property now draws `PolicyEngine/x\y/../microcosm`, which a Markdown renderer escapes (`x%5Cy`, removed by the `..`) and a browser in an `href` reads as `PolicyEngine/x/microcosm`; either way it is refused. Its twin, `PolicyEngine/x\..\..\microcosm`, links no line here in Markdown or in HTML and is not refused. An explicit example pairs the two in a reference definition with a `Dockerfile`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #1146. This is #1146 rebuilt fresh on main, without its New Zealand files: #1282 landed the NZ half on 2026-10-15 with its own stricter grammar, and the NZ hub owns it. Round 2 (
5221cd6a, plus the test-only01da9a37) answers the round-1 review's REQUEST_CHANGES, round 3 (f30da07c) answers round 2's, round 4 (1840bc92) answers round 3's, round 5 (4229cd8c, plus the test-onlyc67030f7) answers round 4's, round 6 (29983852) answers round 5's, round 7 (5f6647b1, plus the test-onlyea912094) answers round 6's, round 8 (25b5bbe2) answers round 7's, round 9 (a0d20514) answers round 8's, round 10 (fa7601cb, plusc36671e6) answers round 9's, round 11 (2822aff1) answers round 10's, round 12 (8a62c938) answers round 11's, and round 13 (4dc0734e, plus the test-only5578c82d) closes a gap of rule 7's own that round 12's review also found, anda738da7d(plus the test-onlye5e14e96) closes one that round 13's review and a fuzz found. Round 14's review approvede5e14e96with no findings; see "Review" below.Invariants
The guard is
test_spec_source_citations.py, with validators intest_support/microcosm_build/spec_source_citations.py. It scans every resource in every country spec package, meaning everybuild/<country>/directory with acountry_package.json(today am, be, nz, uk, us). JSON and YAML resources are parsed, and every string is scanned, mapping keys included. Each URL in a string is first read the way a browser and then GitHub read it. Its authority is parsed as the WHATWG URL Standard parses it. Before the query,\is/, and any number of slashes may followhttps:or start a network-path reference (//host/...). The user (up to the last@) and the port are dropped. The host is percent-decoded, mapped to ASCII as UTS46 (IDNA) maps it, and lower-cased, so%67ithub.com,ｇithub.comanduser@GitHub.com:443are allgithub.com. Then, as GitHub does,%escapes of unreserved characters and of/in the path are decoded,.and..segments resolved, and the query dropped, keeping the#fragment. A URL the standard refuses is read as written. A URL runs to whitespace, so a quote or a bracket inside it is part of it, as it is to a browser. Markdown and prose end a URL at a quote, a parenthesis or an angle bracket, so a string that reads differently that way is read a second time, with each URL ending there. Markdown and HTML decode a backslash before ASCII punctuation and a character reference before a link is followed (x\.py,#L12), so a string holding either is read again with them decoded: as Markdown decodes them and, apart, as HTML does, which leaves a backslash as written (\cis\c, and the URL Standard reads that\as/), each once as text and once as a URL holds what they decode to. A quoted HTML attribute value or a Markdown destination in angle brackets (after the spaces, tabs and one line break CommonMark allows before its<) that starts a URL is read as the browser follows it: decoded, stripped at its ends, and with the whitespace inside it held, so a space in a segment that a later..removes does not end the URL. Where a string holds a Unicode space such as U+00A0, which ends no URL in an attribute or a Markdown destination, its URLs are also read through it. Where a string holds a?and a#, its URLs are also read through the whitespace in their query, which GitHub ignores, sohref=".../x.py?a b#L12"citesx.pyat line 12. The URL Standard deletes ASCII tabs and line breaks from a URL wherever they are, so each of these is read again without them. Every rule below applies to every reading: a citation any reading finds is checked. SoHTTPS://GitHub.com/o/r/blob/<sha>/docs/../%74ools/x%2Epy?plain=1#L12citeshttps://github.com/o/r/blob/<sha>/tools/x.pyat line 12. For those resources:in_repository_line_citations). First, any prefix that names this repository is removed: agithub.com/PolicyEngine/microcosm/(blob|tree|raw|blame)/<ref>/orraw.githubusercontent.com/PolicyEngine/microcosm/<ref>/URL, where<ref>may span segments (main,refs/heads/main,feature/x) and the path after each segment is tried, up to the depth of the deepest path form here, ormicrocosm/(after any../and an optionalPolicyEngine/) before a top-level source directory. The rest counts as in-repository when either holds:packages/,tools/,experiments/,test_support/ordocs/. The anchors are the path from the root and, underpackages/, the path from the distribution directory (microcosm-build/...). A file in a package'ssrc/microcosm/is also named fromsrc/, from themicrocosmnamespace (microcosm/graph/executor.py), from its shard (graph/executor.py), or from a first-level subpackage of its shard (us_runtime/asec_pool.py). Deeper tails do not count, so a foreignutils/loss.pyortests/test_aggregates.pystays foreign however this tree grows. A path here can hold an=(18 tracked files do, such asdocs/evidence/us-import-entry-margins/reconciliation/period=2025-01.json), where a reading starts a name, so the run of path characters (letters, digits,_,.,/,=and-) that ends in a cited name is tried too, and a reverse locator reads a name after an=(lines 3 of a/b=c.py).unpinned_line_citations). A citation is pinned when either holds:citing_commit). The spellings arecommit <sha>,at <sha>,@<sha>, a GitHub/blob/,/tree/,/raw/,/blame/,/commit/or/commits/segment, or araw.githubusercontent.com/<owner>/<repo>/<sha>/URL.<sha>is 7-40 lowercase hex digits with at least one digit and one letter, not running on into a file name, a longer token (-), or an ellipsis (…,...). Aftercommit,ator@it must not run on into a path either: inat a1b2c3d/foo.py:12the hex is a directory. A citation binds to the nearest such commit in its own clause, else the nearest one before it in its string. A clause ends at;or at a sentence end before a capital, bracket or backtick, and a break inside the citation itself (x.py; lines 12-19) does not split it. The fallback to an earlier commit does not apply when the citation's own clause names a moving ref:on/at/@beforemain,masterorHEAD, quoted or not (on `main`), a/blob/,/tree/,/raw/or/blame/segment at one, orraw.githubusercontent.com/<owner>/<repo>/<branch>/, each also asrefs/heads/<branch>. A cited URL at a moving ref counts, socommit a1b2c3d; https://raw.githubusercontent.com/<o>/<r>/main/x.py#L12is unpinned;path/commitmapping, at any depth, whose commit has the same sha form and whose path ends with the cited path, by components. A pin ofpolicyengine_us_data/datasets/cps/cps.pycoverscps.pyanddatasets/cps/cps.py, notarchived_data/datasets/cps/cps.pyorother/cps.py.misspelled_commit_pins). A commit-like token in no recognised spelling is refused when it sits in the citation's clause, or between the citation and an earlier commit it binds to. Examples arearchived 42ed5d45,rev a1b2c3dand(a1b2c3d). The point is that a stray sha cannot sit beside a citation that some other commit pins.unrecognised_line_locators). This is a backstop. A source name followed within 24 characters, with no.,;or line break between, byL<n>,line(s) <n>orl./ll. <n>must start a recognised citation, or that locator must belong to one. Otherwise it is refused (x.py some_function, lines 12-19). A bare number after a colon is left out (x.json: 63 intersectionsis a count).microcosm.<shard>.*name in a resource resolves through import plus getattr. Only names whose second component is a shard in this checkout count (build, calibrate, data, diagnostics, fit, frame, graph), so a format id such asmicrocosm.axiom_input_closure.v1is not read as a symbol.dot_segment_line_citations). A dot segment is..as a whole path segment beside a/, a\or a%2f(GitHub reads%2fas/before it resolves a path), either dot perhaps written%2e, in the string as written or in any source its readings start from (decoded, enclosed or joined). It removes the segment before it, and whitespace, a backslash or markup in that segment can make a renderer remove another segment than a reading does. In[a]( <https://github.com/…/tools/a b/../x.py#L12>)the destination follows a space, and in[a](https://github.com/…/tools/a\b/../x.py#L12)Markdown writes the backslash as%5C: both link to line 12 oftools/x.py. So every line citation in such a string is refused, whichever file it names. Without a dot segment, whitespace or a backslash that a reading takes otherwise than a renderer stays in the segment the renderer follows (%20,%5C), and no source file underpackages/,tools/,experiments/,test_support/ordocs/has either in its path (a test checks), so that link names no source file here.line_links_of_this_repository). A string is refused where any source its readings start from holds this repository's GitHub path,PolicyEngine/microcosm/, with a line fragment (Land a digit) at the first#after it. The source is read as a browser and then GitHub route a path: a backslash as/, as a browser reads it in an HTML attribute, and again as no separator, as a Markdown renderer escapes it (%5C); the dot segments before each?or#resolved, either dot perhaps%2eand an empty segment counted, so a dot segment can form the path (PolicyEngine/x/../microcosm/blob/<sha>/LICENSE#L3); then escapes of unreserved characters and of/decoded and a run of slashes read as one; and the owner and name matched in any case, but not as the end or the start of a longer name (NotPolicyEngine/microcosm,PolicyEngine/microcosm-dynamics). Rule 1 follows a link to the file it names, so where a reading takes a link's ref or authority otherwise than a renderer, it can name no file here, and a commit in the string then pins what it does name: in[a](<https://github.com/PolicyEngine/microcosm/blob/feature+foo/tools/x.py#L12>)a reading starts the name at the+and citesfoo/tools/x.py. Rule 7 needs no file. A renderer's link to a line here names that path, and between a source and the URL a renderer follows lie only the decodings the readings apply, a Markdown renderer's percent-encoding (of a backslash or a tab, among others, which a browser then reads as part of a segment), a browser's steps (a backslash as/, dot segments resolved, and percent-encoding of characters that are neither in the path nor a#) and GitHub's (unreserved escapes decoded, a run of slashes as one, the owner and name in any case). GitHub decodes no%2fin the owner or the name (each such path returns 404), so decoding one here can only add a refusal.The recognised spellings are:
x.py:12,x.py:L12,x.py#L12,x.py L12,x.py line 12,x.py: lines 12,x.py @ L12, and with the number quoted,x.py:"12-19";,,;,—,–or a spaced-), a bracket, andat/on:x.py, lines 12-19,x.py (lines 12-19),x.py [L12],x.py at line 12,x.py — ll. 12-19;lines 12-19 of x.py,line 12 in x.py,L12 of x.py,ll. 12-19 of x.py.Numbers are a line or a range (
12-19,L12-L19, en and em dashes,12 to 19,12 through 19), joined by,orand. The file and the locator may be quoted, backticked or curly-quoted. Only text-source suffixes count (py, yaml, json, toml, sh, md, csv, rs and the like), and a cited file may be anhttps:URL.Scope of each claim:
Out of scope: a dotted name with lines (
build_loss_matrix.create_target_matrix:93-103), because ledger fact keys such asledger.aggregate_fact.v2:5973…read the same way; and a reverse-order locator separated from its file by other words (lines 12-19 of the upstream x.py). The backstop reads forward from a file name only.A bare file name such as
concepts.pycannot be resolved, so rule 2 covers it, not rule 1. A relative path's inner..segments are resolved (tools/x/../y.pyistools/y.py), but a path that climbs above where it starts (../y.py,tools/../../y.py), or an absolute one, is not resolved unless it namesmicrocosm/. A relative reference is read as a path, not as a URL: its backslashes are not read as/, and whitespace ends it, so[a](<tools/a b/../ci_test_plan.py#L12>)citesb/../ci_test_plan.py, which names nothing here; rule 2 refuses it, and since round 11 rule 6 refuses it pinned or not, for its dot segment. Of the 200,000 generated links under "How this was verified", 12,702 relative ones read so, and every one is still refused by another rule.Moving refs are the three branch names
main,masterandHEADin the spellings rule 2 lists. Another branch name (develop), a query ref (?ref=main) orHEAD~1is not recognised, so a citation in such a clause still falls back to an earlier commit.URL reading covers three kinds of URL:
http(s)URLs, with any number of slashes afterhttps:, or none;//host/..., any two of/and\), read as on anhttpspage;github.com/...), before an optional port. A user written before such a host is left as prose, and the host is read from where it starts.A host written with neither is GitHub's only in its ASCII spelling. Another host written that way is read as written, so its query can still hide a locator; the backstop catches a query of up to 24 characters. Punctuation that ends a URL (
.,,,;,:,!,?,),]) is read as prose, and so is a quote or an angle bracket that ends one. Otherwise a URL runs to whitespace, so a quote or a bracket inside it is part of it. The second reading also ends a URL at a quote, a parenthesis or an angle bracket, so a Markdown link destination ([label](url),[label](<url>)), an autolink (<url>), an HTML attribute (href="url") and a URL written straight after another with only a quote between (https://a/x'https://b/...) are each read as a URL of their own. Square brackets end a URL in neither reading, which keeps an IPv6 host ([::1]) whole. Since a citation any reading finds is checked, the second reading can only refuse more. It over-refuses where a quote, a parenthesis or an angle bracket inside a URL whose path names a source file is followed by text that reads as a line locator. In another repository's line URL with text before the#Lfragment (.../x.py?q='v'#L12), the second reading leavesx.py'v'#L12, which the backstop refuses as an unrecognised locator. In a data-source query that quotes a locator-like value (https://www.ons.gov.uk/data.json?where=CLASS='L12'), it leavesdata.json'L12', an unpinned citation ofdata.json. Escaping the character (%27) passes. No live string reads differently the second way. Whitespace is Python's\s, 29 code points: the ASCII ones, U+001C-U+001F, U+0085, U+00A0 and the other Unicode spaces. Bun'sURLreads a URL with any of them inside as one URL at all four places the code-point differential below tries (it drops a tab or a line break and escapes the rest). Since round 10 a URL is also read through whitespace other than ASCII whitespace (tab, line feed, form feed, carriage return and space), and a quoted attribute value or a destination in angle brackets that starts a URL is read whole. Elsewhere a URL broken by ASCII whitespace is read only up to it, as an unquoted attribute value and a Markdown destination outside angle brackets end there too.Hosts are mapped to ASCII with Python 3.14's Unicode database (16.0), plus Unicode 17's U+A7F1 (
iss). A code point added later that a browser maps into ASCII is not mapped.github.com.togithub.com.The decoded readings decode a backslash before ASCII punctuation and character references (decimal, hexadecimal and named; a numeric one with or without its
;), in one pass, as CommonMark decodes a link destination; and, apart, character references alone, as HTML decodes an attribute, leaving each backslash as written. Each is read once as text and once as in a URL, where whitespace, a quote, an angle bracket or a parenthesis that a reference decodes to is percent-encoded, or deleted if it is a tab or a line break. References are read as the HTML standard reads them: one to 0, to a surrogate or past U+10FFFF is U+FFFD, one to a C1 control is the windows-1252 character there where it has one, and one to any other control is that control (Python'shtml.unescapedrops it). In a URL, as in an attribute, a legacy name without its;before a letter, a digit or=stays as written. The readings decode everywhere in a string, including where Markdown leaves an escape as written (a code span, an autolink) and outside any link. So they can refuse more: a regular expression'sx\.py:12is read asx.py:12. No live string decodes differently.The joined readings delete every ASCII tab and line break (U+0009, U+000A, U+000D) from each source above, as the URL Standard deletes them from a URL in an HTML attribute. Outside a URL they join a line's end to the next line's start, so
x.pyending one line and:12starting the next is refused. 16 live strings hold a tab or a line break, and reading them joined changes no rule's output.The enclosed readings read any quoted value after
=(="…",='…', with whitespace around the=) and any destination in angle brackets after](or]:(](<…>),]: <…>), not only in a tag or a link. Each is decoded first, then stripped at its ends (a value of C0 controls and spaces, as the URL Standard strips a URL; a destination of whitespace, as markdown-it strips one), and a value's tabs and line breaks are deleted. One that then starts a URL is read whole, with the whitespace, quotes, angle brackets and parentheses inside it held; any other is left as written. So they can refuse more: prose holdingx="https://github.com/o/r/blob/main/a b/../y.py#L12"is read as a line link ofy.py. No live string reads differently this way.Where a string holds whitespace other than ASCII whitespace (U+000B, U+001C-U+001F, U+0085, U+00A0 and the other Unicode spaces), its URLs are also read through it, to the next ASCII whitespace. No live string holds any.
Where a string holds a
?and a#, its URLs are also read through the whitespace in their query, to the fragment or a line break, three ways: with the query ending at an angle bracket, at a", or at a', what can end each enclosure, so a query does not swallow a later link. So these readings can refuse more: inSee https://example.org/data.csv?x and, much further on in this note, #L12.the query runs to the#, and the URL reads as a line link ofdata.csv, which rule 2 refuses. Two live strings hold a?and a#; neither reads differently this way.Rule 6 refuses a line citation in a string with a dot segment anywhere in it, not only in the cited URL, and another repository's pinned citation too:
x.py:12 at a1b2c3d; see ../docsis refused, and so ishttps://github.com/o/r/blob/a1b2c3d4/a/../x.py#L12. A run of dots that is no whole segment (1 .. 9,x/.../y,sha...) and a lone.are no dot segment. Of the round-10 review's 396 over-refusal probes, the 108 that cite another repository's file through a..are now refused this way. No live string holds a dot segment.Rule 7 reads links that name this repository's owner and name. A relative link (
[a](tools/x.py#L12)) names neither and is left to rules 1 and 2, and a link through a redirect (a renamed repository, a short link) is not followed. It reads prose between the path and a later line fragment as part of the link, sosee github.com/PolicyEngine/microcosm/tree/main/docs and cps.py#L197 at a1b2c3dis refused, as a dot segment's string is. An issue, a comment or a heading (PolicyEngine/microcosm#38,/pull/939#issuecomment-1,#readme) is no line, and a#between the path and a later line fragment (/issues/1 (#2) and x.py#L12) starts the link's fragment first, so neither string is refused. 99 live strings name the repository, and rule 7 refuses none. Since round 13 rule 7 resolves dot segments before it matches the path, as a browser does, so it refuses a link whose path a..forms and not one that a..sends out of this repository (PolicyEngine/microcosm/../other/blob/<sha>/x.py#L1), which rule 6 refuses where a reading finds the citation. A string does not say whether it is Markdown or HTML, so a backslash is read both ways: an HTMLhrefthroughx\y/../microcosm, which linksPolicyEngine/x/microcosm, is refused, as its Markdown twin, which links this repository, must be.A fragment's
%escapes are read as written, so#%4C12is no citation. A character reference in one is decoded (#L12is#L12).Escapes other than unreserved characters and
/stay as written: GitHub reads%23and%3Fas part of a file name (both return 404 as line anchors), sox.py%23L12is no line citation.%2Fis decoded anywhere in the path, which GitHub does only after the ref, so the non-workinggithub.com/PolicyEngine%2Fmicrocosm/blob/...is refused as in-repository.A query is not read: a sha in one neither pins a citation nor counts as a misspelled pin.
A URL's path is resolved only up to the depth of the deepest path form here (10 components today; no tracked file is deeper).
A prose pin followed by
/is no pin, sorepo@<sha>/path/x.py:12is refused: writerepo@<sha> path/x.py:12or a GitHub URL.The guard does not check which repository a commit belongs to. A commit earlier in the string still pins a later citation unless the citation's clause names a moving ref. The live packages use that pattern on purpose: the PUF note names
42ed5d45…once, and later sentences say "at the same commit".The guard does not check that the cited lines say what the prose claims. For this PR that was checked by reading each changed citation at its commit (see "How this was verified"), and a script checks every cited line number against the file at the bound commit.
Each rule has Hypothesis properties, at least one per refusal branch:
an unpinned citation is refused in each of 38 spellings, at any depth, as a value or as a key, and the backstop stays silent on every recognised spelling. Generation draws the reverse spellings, last in the list, least, so since round 9 a worded reverse range (
lines 12 to 19 of x.py) is an explicit example;a covering
path/commitpin passes a citation;a path pin covers exactly the citations its path ends in (a differential against a string-suffix oracle);
an inline commit in any of 12 spellings pins a citation when it comes before it (across clause breaks) or later in its own clause;
a locator quoted in any of 8 spellings, with any of 8 quote marks (or none) before and after the file and the numbers, is one citation, as a value or a key, at any depth: unpinned for a foreign file, in-repository for any of the 19 in-repository forms;
a commit only in a later clause, after
;,.,!or?, does not pin;two citations in separate clauses each bind their own commit;
an earlier commit does not pin a citation whose clause names any of 12 moving refs, or a citation whose cited file is a URL at one of 5 moving refs (blob, raw and
refs/heads/forms), across each clause break;a commit named in another string does not pin;
none of these names a commit: a decimal, an all-letter word, a prefix of 6 hex digits or fewer, a 41-64-hex digest, a branch name, a truncated
<sha>…or<sha>..., or<sha>-dirty;a hex directory after
commit,ator@(at <sha>/x.py:12, at any depth under it) names no commit, leaves the citation unpinned and is no misspelled pin;a pin of another file, or a pin at a branch, does not cover a citation;
a commit-like token in no pin spelling is refused in the citation's clause or between the citation and its binding commit, and is not refused in another clause;
symbol citations are neither line citations nor backstop hits;
a symbol between a file and its lines is refused by the backstop;
an in-repository path, in 19 forms including GitHub URLs (at
main, a sha,refs/heads/mainandfeature/citations),raw.githubusercontent.com,microcosm/tools/...and../microcosm/..., is refused even when pinned inline or bypath/commit;a URL of this repository at a ref of 1-4 random segments, under 5 URL prefixes, is refused even when pinned inline;
a URL of this repository is refused however a browser and GitHub let it be written. Reading the citation's text again changes nothing. The property varies:
https://,http://,https:,https:/,https:///,https:\\,http:\/,//,\\,/\,///or none);:0443, a user, a user and password with a port, or two@s);%-escaped as UTF-8;%-escaped in upper or lower hex (/only after the ref), up to 6 of its/written\, and any of 11 dot-segment detours, four with a quote or a bracket in the segment removed ('x'/../,"/../,<x>/../,`x`/../);#Lfragment.Since round 7 every full-width, escape and backslash index drawn lands on the host or a
/(it wraps around their length). Sinceea912094a dot at a full-width index is written as any of．,。and｡, which the standard reads as.. Over 2,000 examples drawn with its strategies atea912094, 1,490 put a full-width character and 1,467 an escape inside the host, 213, 151 and 121 write a host's dot as．,。and｡, 969 add a trailing dot, 1,637 write a separator as\, 688 take a detour with a quote or a bracket, 184 put one in the query, and every lead and authority appears 184-727 times;a random query of up to 120 characters, quotes and brackets included, hides no line citation in another repository's URL, pinned at a sha or unpinned at
main;a line link of this repository is refused however Markdown or HTML escapes it. The link is under 5 URL prefixes at a sha, to 3 files at
#L<a>or#L<a>-L<b>. It is set in 7 ways: an inline link, a pointy destination, a reference definition, a link with a title, and a double-quoted, single-quoted or unquotedhref. 1-24 of its characters are escaped, each as a backslash (Markdown only), a decimal reference with 0-3 leading zeros, a hexadecimal one or a named one, and in anhrefup to 3 tabs or line breaks go anywhere as references, and in a quoted one also as written. Since round 10 the link may go through a segment that a later..removes, holding whitespace the setting holds (a no-break space anywhere; in a quotedhrefor a pointy destination also a space, and in a quotedhrefa form feed, a tab or a line break), and in HTML up to 2 of the slashes after the authority are written\, with a reference after each. An explicit example puts a decimal reference after a backslash in a double-quotedhref, the round-9 review's shape (300 examples; in one run 47% escaped three or more characters, 25% put tabs or line breaks in anhref, 81% took a detour and 18% wrote backslashes);every line citation in a string with a dot segment is refused, pinned or not, and without it the same string's citations pass rule 6. The dot segment is one of 8 spellings (
.., either dot%2e,..,..,\.\., or a tab between the dots) beside a/, a\or a%2fin either case, at 7 places around a citation of another repository; in its place goes one of 7 near misses (.,...,x,%2e,.x,x..,.). The string is a value or a key at depth 0-2 (300 examples);a line link of this repository is refused whatever whitespace its query holds: spaces, tabs, U+00A0 and U+3000 among
ab=&+%, up to 60 characters, in 8 enclosures (200 examples);a line link of this repository is refused whatever its ref or authority holds, and its twin in another repository (
PolicyEngine/microcosm-dynamics,PolicyEngine/microcosms,NotPolicyEngine/microcosm,PolicyEngine/micro) is not, nor is one through a..that leaves it (PolicyEngine/x/../../microcosm,PolicyEngine/x%2f..%2fmicrocosm,PolicyEngine/x\..\..\microcosm). The ref is 1-3 segments of letters, digits and_+@!$,;=()-; the authority one of 5 (github.com, a user, and whitespace in a user before an escaped host, a full-width one, or one with an ideographic full stop and a port); the repository one of 9 spellings a browser and GitHub route (case, escapes, a doubled slash, a.segment, and four through a..:x/../,//x/%2e%2E/../,a%2fb/../andx\y/../, which a Markdown renderer escapes and a browser in anhrefreads asx/y/../, linking no line here, and which is refused either way); the routeblob,treeorblame; the file one of 3 source files or one of 3 whose names no reading takes for a source (LICENSE, aDockerfile,SHA256SUMS); the query none,?plain=1, 30 letters, whitespace or?a/../../../../..; in 8 enclosures (300 examples);on a temporary tree, a 500-segment ref is searched only as deep as the deepest path there (4
is_filechecks), and the deepest path still resolves;on a temporary tree, exactly the anchored tails of three files resolve;
the scan for source names finds exactly the matches
finditerfinds. The forward locator and the backstop are tried only where a match can start (the start of a run of path characters,http(s):, or where the previous match ended), and since round 8 the reverse locator once per list of line numbers, from the list's head rather than from each number in it. Since round 9, when it fails there, it is tried once more from the list's last number, where a range can end and another start (L1-L2-3 of x.py). The text is random path pieces, whole locators, list pieces,https:, quotes and breaks (500 examples; in one run 37% held a forward match, 29% a reverse one and 45% a backstop near match), plus examples that need the previous match's end,https:inside a run, a reverse locator after a failed list, lists mixingLnumbers, labels andand, and four chains of ranges;the locators, now reading whitespace, quotes and numbers possessively and finding a spaced dash by a lookbehind, match exactly what round 7's patterns (kept in the test) match, the same spans and the same file names, over random text of list pieces, separators, dashes, quotes, labels and source names (1,000 examples; in one run 47% held a forward match and 30% a reverse one), plus eight built around dashes with and without spaces, quoted lists,
ANDand chains of ranges;the bindings, misspelled pins and backstop hits found by bisection equal those of the whole-string scans they replace, which the test keeps as the reference: random text of citations, pins in each spelling, stray shas, moving refs and clause breaks (500 examples, plus five built to tie the nearest commits on both sides, end a clause break where a citation starts, put a commit inside a cited URL, fall back past a later clause's moving ref, and mix backstop hits with citations);
eight hostile strings are scanned in under 2 s of CPU time: 60,000-character runs of
:,//,aandgithub.com/, 20,000 escapes afterhttps://a/, and strings of 4,000-8,000 citations (unpinned, pinned in their clause, pinned by a commit clauses before, and mixed with backstop hits). Here they take 0.2 s; the round-6 helper takes 92 s on the first five alone. Since round 8 nine more strings must fit in the same 2 s, the exponential ones first, so a regression fails before the slower ones run. Each took 4.6 s or more atea912094, two over 120 s, and together they take 0.03 s here (see the table under Review). Since round 9 nine more must fit: a failed range with 32,768 spaces after its dash or itsto, a label with 65,536 spaces after it, 16,000 ranges each ending where the next starts, a list read again from its last number, and runs of�,&#x,\and&a. Since round 10 seven more, in 2 s of their own: runs of="https:,](<https:and&followed by a 28-letter name, a quote followed by 100,000 letters, a destination of 50,000 escapes, and URLs through 100,000 no-break spaces (0.8 s here under load, each linear in its length). Since round 12 ten more, in 2 s of their own, each of 52,000-64,000 characters: runs of this repository's path, of that path with a line fragment, of#L1,/./,/,%2f..%2fand=..%2f,a=before a citation,=x.py:1, anda=afterlines 1 of(0.23 s here; quadrupling each quadruples its time). Since round 13 five more in those 2 s:PolicyEngine/then 10,000x/and 10,000../, and runs of/..,?/..,%2e%2E/and\..(0.13 s here; quadrupling each quadruples its time);Differential tests:
uk_data_target_parity.jsoncitation binds to is its ownincumbent_commit;uk_population_targets.jsoncitation binds to is a prefix of itsregistry_parity.pinned_ref;source_stages.jsonand the generatedsources.yaml;loss.pybinds to2e044571and itscps.pyto42ed5d45;ASEC_CENSUS_PERSON_COLUMN_NAMES;URL(14 hosts read, 6 the standard refuses).Two local differentials, against bun itself, cannot run in CI, which has no bun (scripts and results in the session's review folder):
Every code point. Each of the 1,112,064 code points was escaped into a host (
a<cp>b.org) and read by both:ß,ς, and letters Python's Unicode 16 does not case-fold yet), so none could be GitHub's.Before U+A7F1 was mapped, it was the one code point the standard maps into plain ASCII and the reader did not.
Every code point at four places in a URL of this repository (round 7): in a user (
https://u<cp>v@github.com/...), inside a dot segment (tools/a<cp>b/../ci_test_plan.py), as a dot segment (tools/<cp>/../) and in a query (?v=<cp>w), each followed by#L12. At each place 1,112,035 code points agree, 0 are read as this repository's only by the reader, and the other 29 are the whitespace code points, where the URL ends by Scope.26,350 generated URLs. They cross 18 leads, 8 users and 11 ports with host spellings (escapes, full-width and look-alike letters, ignorable code points, ideographic dots, trailing dots, bad escapes), 9 paths and 10 queries and fragments:
?or#, or no path.Why
The US and UK packages cited sources by line numbers that drift. The in-repository ones drifted under ordinary edits, and the external ones had no commit, so they drifted with the other repository's main. On main (
90ff43f9) this guard flags US 45 unpinned + 18 in-repository citations and UK 47 unpinned; am, be and nz are clean. With this branch all five packages scan clean under all seven rules.Changes
US,
ecps_parity_known_gaps.json(6hermetic_build_contractstrings andis_unmarried_partner_of_household_head.evidence.semantic_non_substitutes.derived_A_EXPRRP; main dropped the employer-premium gap that #1146 also edited):asec_pool.py lines 61-75microcosm.build.us_runtime.asec_pool.load_asec_h5_tables(plus_prepare_year_inputwhere the restore matters)asec_pool.py lines 397-422microcosm.build.us_runtime.asec_pool._with_relationship_recodetools/build_us_puf_support_base.py lines 85-88 and 659-699_parse_args (--asec-h5),_pooled_asec_sources_from_args,_asec_sources_from_argsbuildj_base.sh lines 65-69/65-81/70-81 and 103-108--asec-h5 2024=, 2023=, and 2022=(and--puf-h5) argumentsbase_j.summary.json lines 55-75/317-318base_source.sources[].sha256,puf_sha256, andbase_source.metadata.sources[].sharefor the equal shares#720 made several of these sentences false as written, for example "reads the prebuilt person tables without refreshing omitted Census columns". The pool now restores the reviewed Census person columns, so each sentence says which columns the restore leaves out. Tests check:
ASEC_CENSUS_PERSON_COLUMN_NAMES;No exclusion decision or
reasonchanges.US,
source_stages.json→ regeneratedspec/sources.yaml(8 notes): every archived us-data citation names commit42ed5d45c56df80d754fbe24cce21cfeb8d05cbeand a package-relative path (datasets/cps/cps.py,datasets/cps/extended_cps.py,parameters/take_up/wic_takeup.yaml,utils/randomness.py). That includes the employer-premium note added since #1146. Round 2 also moves the PUF note's commit ahead of its first two citations: it now reads "The pinned PUF lineage (archived commit 42ed5d45…) … (datasets/puf/puf.py lines 1378-1398) … (datasets/puf/disaggregate_puf.py lines 55-60 and 85-123)". Before, it named the commit only sentences later, which binding would not accept.US,
target_parity_manifest.jsonandtools/build_us_target_parity_manifest.py:utils/loss.py:67-70 at 42ed5d45…. The ESI origin's CPS citation now readsarchived commit 42ed5d45 datasets/cps/cps.py lines 197-271 and 1575-1581. It had readarchived 42ed5d45 cps.py L197-271/L1575-1581, which round 1 counted as pinned by the unrelatedcommit 2e044571earlier in the string. Both ranges were re-read at42ed5d45: 197-271 is the ESI imputation block (ESI_POLICYHOLDER_VARIABLEthroughimpute_employer_sponsored_insurance_premiums's return), and 1575-1581 is the policyholder flag and premium assignment.UK:
uk_data_target_parity.jsonand its generatoruk_runtime/data_target_parity.py: all 13 uk-data citations name the register's ownincumbent_commit,a43e8c5. Main moved the register from Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146's8629dbbftoa43e8c5, so each one was re-read ata43e8c5, not ported.uk_population_targets.json: 41 strings nameregistry_parity.pinned_ref,12a1e028afeef08d8b2d74ee03fd9de3a78b2dd3, withtargets/-qualified paths. 23 carry Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146's rewrite verbatim, because main has not touched them since. The other 18 were added or reworded on main after Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146, so each was read at12a1e028and pinned there (for examplecompute_esais atincome.py98-102 at that ref, not 115-119).Wrong pointers corrected. Each cited range was read at its commit. These seven pointed at other code:
ssi_take_upnotesssi_reportedanchor; the save is 759-766scf_wealthnotesSCF_PREDICTORS168-177hmrc_cgthmrc_cgt.py:195_band_targets)hmrc_spihmrc_spi.py:230defget_targets)datasets/imputations/frs_only.pyservices/services.py:137-148(thedfe_education_spendingimputation)sources/_land.py:52-73exceptexcept Exceptionintargets/sources/hmrc_salary_sacrifice.pylines 34-109, which logs the error and returns only the contributions targetSmaller fix:
prior_year_incomegains the predictor-definition rangeextended_cps.py 234-248, matching its sibling stage.Tests whose literal assertions moved. Three exclusion tests asserted
"buildj_base.sh lines 65-69". They now assert the content anchors and check thatbase_j.summary.jsonholds three 64-characterbase_source.sources[].sha256values.Re-pins
Every byte edit to
source_stages.jsonmoves these (round 2's PUF-note edit moved them again):90ff43f9)source_stages.jsonraw sha256 (sources.yamlstage_asset, generatorFROZEN_LEGACY_RESOURCE_SHA256,us_spec_bundle.py,test_us_bundle_core_contracts.py)dc3b7298…5fc76f9b…spec_sha256(us-f0-coverage.json×2,test_us_multispine_pool_tool.py)c677d50b…b5c87a41…documentation_sha256(us-f0-coverage.json)801a99da…811b4091…EXPECTED_HASHES.source_manifest87b1e124…10c05f0f…EXPECTED_HASHES.late_resource_semantics531b48cd…6634f751…EXPECTED_HASHES.full_checkpoint030c8422…c88b0d7f…us-f0-coverage.jsonpool-code observed shaaf44d502…bbd77902…How they were produced:
sources.yamlwas regenerated withtools/generate_us_bundle_from_constants.py; its diff is the rewrapped PUF note and the new raw sha, and--checkexits 0.build_inventory_coverage(load_bundle("us")): before the re-pin exactly these three items failed (38/41), each with only its digest differing.us-f0-coverage.jsonwas regenerated withtools/spec_engine_coverage.py: 42235/42235 configuration fields and 41/41 inventory checks. Exactly ten leaves moved: the expected and observed copies of the three inventory digests,documentation_sha256, bothspec_sha256copies and the pool-code observed digest.--checkpasses.dc3b7298,c677d50b,801a99da,87b1e124,531b48cd,030c8422,af44d502) and round 1's (e2de2b41,ca232744,3cb60930,ecf9ca2b,e8797144,189d4e28,20e3cd3b). The one hit ise2de2b41…, round 1's truncated-hash counterexample, quoted as such in the helper's docstring, a test and the changelog.legacy_json, so only the unpinned UK package fingerprints move.uk_data_target_parity.jsonequalsbuild_uk_data_target_parity()after a JSON round trip.Operational cost: the stacked US pool checkpoint identity binds the late producer resource semantics, which hash
source_stages.json's bytes. Existing stacked US pool stage checkpoints will therefore be treated as stale and rebuilt on the next build, as after any source-stage note edit. No behaviour or data changes.How this was verified
Every citation this rebuild wrote beyond Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146 was read at its commit in local clones of policyengine-us-data and policyengine-uk-data. That covers:
a43e8c5rather than ported from Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146's8629dbbf;12a1e028;The strings carried over from Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146 verbatim rest on Cite US and UK spec sources by symbol or at a commit, never by a drifting line #1146's audit: as its body reports, proposers read each citation at its commit and separate verifiers re-derived every edit.
After merging main, a script checked all 237 US and UK line citations mechanically, taking for each one the commit it binds to under rule 2 (or its covering path pin). In every case the commit resolves in one of the two clones, the cited path resolves to exactly one file at that commit, and every cited line number is within the file. 237 pass, 0 out of range, 0 unresolved. NZ's 46 citations are all covered by
path/commitpins to the ops repository.The guard, run against main (
90ff43f9) with this branch's helper: US 45 unpinned + 18 in-repository, UK 47 unpinned, am/be/nz 0. The 4 US citations beyond round 1's 41 are the PUF note's two, in both resources, which binding now refuses. Against this branch: 0 for every rule in every package, including main's latest UK resources.Round 2's rules, measured on the tree before adopting each one:
Round 5's URL reading, measured on the tree: it rewrites 69 live strings, all queries on data-source URLs (ArcGIS and ONS geography services, NISRA, Nomis, data.census.gov, and state revenue and legislature downloads), and none of them carries a line citation. A differential scan, the helper at
1840bc92against4229cd8cover every resource, finds 0 classification changes across 200 resources and 283 citations.Round 6's authority reading, measured on the tree: it reads every one of the 4,482,233 live strings exactly as round 5 did. Round 6 changes only how URLs are read (the helper's docstring, imports, URL constants and readers), so no classification changes.
Round 7, measured on the tree: it reads all 4,482,233 live strings exactly as round 6 did, and a differential scan, the helper at
29983852against5f6647b1, finds 0 changes in any rule's output (citations, bindings, misspelled pins, backstop hits, in-repository hits) across 200 resources and 283 citations.Round 8, measured on the tree: the helper at
ea912094against25b5bbe2over every resource, 200 resources, 4,482,233 strings and 283 citations. 0 strings read differently the first way, 0 are read a second way, and every rule's output is unchanged. The scan took 89 CPU seconds, against 96 atea912094.Round 9, measured on the tree: the helper at
25b5bbe2againsta0d20514over every resource, 200 resources, 4,482,233 strings and 283 citations. Every string's round-8 readings come first and unchanged, no string decodes differently, and every rule's output is unchanged. 16 strings, all UK (source_stages.jsonandspec/sources.yaml), hold a tab, and each gains a joined reading, which changes no rule's output. The scan took 121 CPU seconds, against 101 at25b5bbe2.Round 9's reverse scan against
finditer: 200,000 random strings ofLnumbers, plain numbers, dashes,to,through, separators, labels, quotes andof/inendings, 0 differences.Round 9's decoded readings against a renderer: 200,000 random escapings of four line links of this repository (a sha,
main, a network-path reference and a relative path), each set as an inline link, a pointy destination, a reference definition or a double- or single-quotedhref. markdown-it-py 4.2 rendered each Markdown setting and HTMLParser read eachhref. 169,582 rendered to a link this guard refuses, and every one of them is refused as written. The rest render to no line link: an escape without its;followed by a digit (1then2) is another number.Round 10, measured on the tree: the helper at
a0d20514againstc36671e6over every resource, 200 resources, 4,482,233 strings and 283 citations. Every string's first reading is unchanged, no string gains a reading, and every rule's output is unchanged, as is whether each cited path resolves here. 1,226 strings hold a&or a backslash and are decoded the new ways, each to a reading round 9 already had. No live string holds a Unicode space or reads differently through an enclosed value. The scan took 137 CPU seconds, against 136 ata0d20514.Round 9's review evidence replayed at round 10: all 45 of its mixed-context variants (29 render to a line link of this repository) and all 8,281 targets of its 27,625-input matrix are refused, and no input it found refused passes now. 17 more inputs are refused, strings like
\#L12that render to no line link, which the backstop refuses.Round 10's readings against renderers: 200,000 generated links. Five line links of this repository (at a sha, at
main, a network-path reference, a deeper file and a relative path) were each set in 7 ways (an inline link, a pointy destination, a reference definition, a link with a title, and a double-quoted, single-quoted or unquotedhref), escaped at a rate of 0-30% (a backslash in Markdown, a decimal reference with 0-3 leading zeros, a hexadecimal one or a named one). Half took a detour through a segment holding one of 10 whitespace code points, written or as a reference, that a..removes. In HTML, half wrote 1-2 slashes as\, often with a reference after one, and 30% put in a tab or a line break. Values and pointy destinations were padded at their ends, and=sometimes had whitespace around it. markdown-it-py 4.2 rendered the Markdown, parse5 7.3 parsed the HTML, and bun 1.3 resolved each link. A link counted when bun's parsed URL was a GitHub path to a file in this checkout with anL<n>orL<a>-L<b>fragment. 113,469 rendered to such a link. Every one written as an absolute URL is refused as written. 12,702 written as relative references are not, the Scope limit, and each of those is still refused by another rule. A first run of 2,000 found five shapes no reading refused, fixed inc36671e6: an attribute value starting or, and a destination in angle brackets padded with spaces, U+3000 or .Round 11, measured on the tree: the helper at
c36671e6against2822aff1over every resource, 200 resources, 4,482,233 strings and 283 citations. Every string's first reading is unchanged, no string gains a reading, and every rule's output is unchanged, as is whether each cited path resolves here. Two strings hold a?and a#, and their query readings are readings they already had. No string holds a dot segment, so rule 6 refuses nothing. The scan took 127 CPU seconds, against 122 atc36671e6.Round 10's review evidence replayed at round 11. Its 607 generated contexts, 51 more it left unclassified and 21 family probes each render to a line link of
tools/ci_test_plan.pyin markdown-it-py 4.2, parse5 7.3 and bun 1.3, and no rule refused any of them atc36671e6. All 679 are refused now: rule 6 refuses each, and rule 1 also refuses 128. All 8,281 targets of its 27,625-input matrix and all 45 of its mixed-context variants are refused, and no input's refusal changes. Of its 396 over-refusal probes, pinned citations of another repository in prose, attributes and destinations, the 288 without a dot segment are refused exactly as atc36671e6(4 of them), and the 108 with one are now refused by rule 6, as Scope states.Round 11's readings against renderers. Round 10's 200,000 generated links, rerun with rule 6, give the same counts: 113,469 render to a line link of this repository and each is refused as written, and each of the 12,702 relative ones is still refused by another rule. The round-11 counterexample test's 25 inputs (11 shapes of the four findings and 14 queries holding whitespace) each render to a line link of
tools/ci_test_plan.py, and 2,000 inputs drawn as the query property draws them each render to a link to the file at their lines. The guard refuses every one.Round 11's timing strings, runs of what a dot segment, a URL read through its query and a destination after whitespace start with, grow linearly: doubling each doubles its time. A query reading that requires a query, or a fragment, takes 14.5 s on
"https:" * 20_000 + " ?#", against 0.02 s; the timing test runs that string.Round 12, measured on the tree: the helper at
2822aff1against8a62c938over every resource, 200 resources, 4,482,233 strings and 283 citations. Every reading and every earlier rule's output is unchanged. No citation's path run differs from its file, no string holds a dot segment with or without%2f, and rule 7 refuses none of the 99 strings that name this repository. The scan took 116 CPU seconds, against 115 at2822aff1.Round 11's review inputs at round 12. 22 inputs, its two shapes and their variants (7 refs in Markdown and HTML, 5 authorities after a code span's
=", a long query, an escaped owner with a doubled slash, and%2f..%2f), each render to a link to line 12 oftools/ci_test_plan.pyin markdown-it-py 4.2, parse5 7.3 and bun 1.3. No rule at2822aff1refused any of them; rule 7 refuses all 22, and rule 6 the%2fone too.Rule 7 against renderers. The property's space, 8 enclosures by 5 authorities by 9 repository paths by 4 queries by 2 refs, is 2,880 inputs. Of the 1,600 that name this repository, 1,400 render to a line link of it and all 1,600 are refused; the other 200, a tab and a space in a user before a full-width host in a Markdown destination, render to no URL. None of the 1,280 that name another repository is refused. 2,000 inputs drawn as the property draws them render to 1,786 line links of this repository, and all 2,000 are refused, their twins in other repositories none.
Round 13, measured on the tree: the helper at
8a62c938againsta738da7dover every resource, 200 resources, 4,482,233 strings and 283 citations. Every reading and every earlier rule's output is unchanged (0 differences), no string holds a dot segment, and rule 7 refuses none of the 99 strings that name this repository. The scan took 117 CPU seconds, against 118 at8a62c938.A dot segment that forms this repository's path, against renderers: 3,960 inputs, 4 files (
LICENSE, aDockerfile,SHA256SUMSandtools/ci_test_plan.py) by 22 paths (15 that a browser and GitHub route here, such asPolicyEngine/x/../microcosm,x/%2E./,//x/../../,a%2fb/../,x\..\and.., and 7 that route elsewhere) by 5 enclosures by 3 authorities by 3 queries. 1,920 render to a line link of this repository in markdown-it-py 4.2, parse5 7.3 and bun 1.3. At8a62c938rule 7 refused 264 of them and the rules together 678, so 1,242 passed. All 1,920 are refused now, and none of the 7 paths that route elsewhere. Rule 7 also refuses 780 inputs that render to no line link here, each naming this repository's path once its dot segments are resolved: 540 bare URLs, which markdown-it does not link; 180 Markdown destinations that the space in a user ends; and 60 Markdown links throughx\..\, where Markdown reads\.as an escaped dot and HTML keeps the backslash.Rule 7's property space at
e5e14e96, rendered the same way: 8 enclosures by 5 authorities by 16 repository paths by 5 queries by 2 refs by 2 files, 12,800 inputs. Of the 7,200 that name this repository, 6,000 render to a line link of it and all 7,200 are refused; of the other 1,200, 900 are the tab and space in a user before a full-width host in a Markdown destination, which render to no URL, and 300 arex\y/../in anhref, which a browser reads asPolicyEngine/x/microcosm(the over-refusal under Scope). None of the 5,600 that name another repository or leave this one is refused, and none routes here. 2,000 inputs drawn as the property draws them render to 1,695 line links of this repository; all 2,000 are refused, their twins none, and no twin routes here.GitHub's routing, by
curl --path-as-ison 2026-10-28:PolicyEngine/x/%2e%2e/microcosm/blob/<sha>/LICENSEandx/.%2e/return 200 (a browser has resolved both before it asks);PolicyEngine/x%5Cy/../microcosmand..%5Cx/../microcosmreturn 200 (GitHub readsx%5Cyas one segment, which the..removes);PolicyEngine/x%2f..%2fmicrocosm,x/..%2fmicrocosm,x/%252e%252e/microcosm,x%5c..%5cmicrocosmandx/y/../microcosmreturn 404.Rule 7 fuzzed against the same renderers, at
4dc0734eand ata738da7d. A first fuzz, 20,000 links with paths of random tokens, found 3 of its 385 line links here that rule 7 missed at4dc0734e, each a Markdown destination holding a backslash. A second, 40,000 inputs, builds detours (junk segments, then as many parent segments) through many spellings of.,..,/,\,?and#(escapes, references, Markdown escapes, tabs), mixed with token soup, in 9 enclosures (inline, pointy, reference definitions, autolinks, bare, and threehrefquotings) and 6 authorities. 5,590 render to a line link here; rule 7 at4dc0734emissed 58, each a Markdown link with a backslash, and ata738da7drefuses all 5,590, and all 385 of the first fuzz. Two more seeds of the second ata738da7dbring it to 60,000 inputs, 8,407 line links here, all refused.Tests
Run locally after merging main, one file at a time,
-n 0,-p no:cacheprovider(no full suite). All pass:After merging
90ff43f9:tools/generate_us_bundle_from_constants.py --skip-validation --checkandtools/spec_engine_coverage.py --check(at4229cd8c; since thenc67030f7,29983852,5f6647b1,ea912094,25b5bbe2,a0d20514,fa7601cb,c36671e6,2822aff1,8a62c938,4dc0734e,5578c82d,a738da7dande5e14e96change only the guard's test file, its helper, which nothing else imports, and the changelog fragment): exit 0; coverage 42235/42235 configuration fields, 41/41 inventory checks.test_spec_source_citations.py: 137 passed ate5e14e96(218 s; 31 of 31 properties reached their limits, 8,600 passing examples)test_us_plan.py: 36 passedtest_country_spec.py: 136 passedOn round 2, before merging
90ff43f9. Main's newer commits touch UK resources,uk_runtime,gates.pyandus_runtime/worker_identity.py, and CI runs these files on the merged head:test_us_bundle_core_contracts.py: 17 passedtest_us_spec_bundle.py: 4 passedtest_spec_engine_country_bundles.py: 21 passedtest_spec_engine_reemission.py: 8 passedtest_spec_engine_loader.py: 10 passedtest_spec_engine_inventory_coverage.py: 13 passedtest_us_scf_wealth.py: 31 passedtest_us_relationship_inputs.py: 17 passedtest_us_parity_reference.py: 17 passed, 1 skippedtest_us_other_health_insurance.py: 27 passedtest_us_financial_assistance_exclusion.py: 3 passedtest_us_survivor_benefits_exclusion.py: 3 passedtest_us_wic_nutritional_risk_exclusion.py: 4 passed, 1 skippedtest_us_dc_ptc_take_up_exclusion.py: 4 passedtest_us_early_head_start_exclusion.py: 6 passed, 5 skippedtest_us_medicare_take_up.py: 19 passedtest_us_asec_sources.py: 41 passedtest_release_input_coverage.py: 86 passedtest_release_target_parity.py: 37 passedtest_us_esi_premiums.py: 148 passedtest_us_chronicle_feed.py: 27 passedtest_us_soi_congressional_district_vintage.py: 51 passedtest_us_soi_state_agi_bands.py: 29 passedtest_us_multispine_pool_tool.py: 31 passedSkips are the files' own data-gated cases. The engine-contract copies and the UK and NZ suites ran in round 1 and CI, and were not rerun locally for round 2. Round 2 changes no UK or NZ resource, and no UK or NZ test names a file it changed. The one shared module it changes,
inventory_coverage.py, moves only three US digests. CI runs every suite.Mutants
Each mutant weakens one rule, or one resource, and runs the guard tests named by its
-kexpression (-n 0 -x). Files were restored byte for byte after each run. Since round 5 they run in a separate clone of the branch, so a mutant can never reach the branch itself. Since round 6 every run starts with Hypothesis's example database cleared, so no mutant is killed by a counterexample saved from another. The harness first runs the unmutated tests over the union of those expressions and stops if they fail. It counts a mutant as killed only when a test fails, with no setup or collection error. Until the round-10 run finishes, the table shows it as far as it has reached, the round-7 run atea912094for the rest of M1-M77, and pending for the rest. The round-8 run at25b5bbe2was stopped at M89 when round 8's review arrived; of M1-M89 it killed all but M24, M80 and M81 (see Review, round 8). The round-9 run ata0d20514killed each of the 37 mutants it reached before round 9's review arrived. The round-10 run atc36671e6reached M93-M128, then M1-M38, and killed each but M12; it was stopped there. The round-11 run, 150 mutants at2822aff1, ran with Hypothesis limited to each test's explicit examples and killed all 150. The round-12 run, 167 mutants at8a62c938(M131-M135 re-anchored to the new dot-segment pattern, M147-M163 new), ran the same way in three clones at once; the 8 properties with no explicit example were skipped. It killed all 167. Round 13 changes only rule 7's reading, so round 13 reran rule 7's mutants the same way. At4dc0734e, 16 of them, M147-M157 (M149-M152 re-anchored to its single reading) and M164-M168 new, were all killed. Ata738da7d, 17, with M149-M151, M164, M167 and M168 re-anchored to its two backslash readings and M169 new: all 17 were killed (e5e14e96adds only test inputs). The table shows thea738da7drun for rule 7's mutants and round 12's for the rest:test_a_decimal_a_word_a_digest_a_truncation_or_a_branch_names_no_committest_a_commit_before_a_citation_or_later_in_its_clause_pins_ittest_a_commit_named_in_another_string_does_not_pin_a_citationtest_a_pin_of_another_file_or_at_a_branch_does_not_cover_a_citationtest_a_path_pin_covers_exactly_the_citations_its_path_ends_intest_a_line_citation_of_this_repository_is_refused_even_when_pinnedtest_in_repository_paths_resolve_from_an_anchor_or_a_repository_prefixlines 12-19 of x.py) is droppedtest_an_unpinned_line_citation_is_refused_at_any_depthtest_an_unpinned_line_citation_is_refused_at_any_depthtest_only_names_under_a_shard_are_symbol_citationstest_a_commit_only_in_a_later_clause_does_not_pin_a_citationtest_a_commit_before_a_citation_or_later_in_its_clause_pins_ittest_a_commit_only_in_a_later_clause_does_not_pin_a_citationtest_a_commit_only_in_a_later_clause_does_not_pin_a_citationtest_a_commit_in_no_pin_spelling_beside_a_citation_is_refusedtest_a_commit_in_no_pin_spelling_beside_a_citation_is_refusedtest_a_commit_like_token_in_another_clause_is_no_misspelled_pintest_in_repository_paths_resolve_from_an_anchor_or_a_repository_prefixtest_in_repository_paths_resolve_from_an_anchor_or_a_repository_prefixtest_a_path_pin_covers_exactly_the_citations_its_path_ends_in…,...,-x) pinstest_a_decimal_a_word_a_digest_a_truncation_or_a_branch_names_no_committest_a_line_locator_in_an_unrecognised_spelling_is_refusedtest_no_resource_cites_a_line_in_an_unrecognised_spelling[uk]to/throughrangestest_an_unpinned_line_citation_is_refused_at_any_depthl./ll.labels and upper-caseLINESare droppedtest_an_unpinned_line_citation_is_refused_at_any_depth/raw/,/blame/and/commit(s)/segments no longer pintest_a_commit_before_a_citation_or_later_in_its_clause_pins_ittest_a_break_inside_a_citation_does_not_end_its_clausetest_an_earlier_commit_does_not_pin_a_citation_whose_clause_names_a_moving_reftest_the_round_two_review_counterexamples_are_refused#locator no longer accepts a quote before itsLnumbertest_a_quoted_line_locator_is_refused_as_a_value_or_a_keycommit,at,@) may run on into a pathtest_the_round_two_review_counterexamples_are_refusedtest_the_round_two_review_counterexamples_are_refusedraw.githubusercontent.comURL at a branch is no moving reftest_the_round_two_review_counterexamples_are_refusedrefs/heads/<branch>ref is no moving reftest_an_earlier_commit_does_not_pin_a_citation_whose_clause_names_a_moving_reftest_the_round_three_review_counterexample_is_refusedtest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_the_round_four_review_counterexamples_are_refusedtest_a_query_does_not_hide_a_line_citationtest_a_url_of_this_repository_is_refused_however_github_lets_it_be_written/is not decodedtest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_the_round_four_review_counterexamples_are_refusedtest_a_long_ref_is_searched_only_as_deep_as_the_deepest_path_heretest_a_long_ref_is_searched_only_as_deep_as_the_deepest_path_here%23and%3Fincludedtest_the_round_four_review_counterexamples_are_refusedtest_the_round_four_review_counterexamples_are_refused.segment is kepttest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_a_url_of_this_repository_is_refused_however_github_lets_it_be_written//host/...) are not readtest_the_round_five_review_counterexamples_are_refused\is not read as/test_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtenℌstaysH)test_a_url_is_read_as_the_url_standard_reads_it[https://git\u210cub.com/o/r/blob/a1b2c3d/x.py#L12-https://github.com]test_a_url_is_read_as_the_url_standard_reads_it[https://gi%C2%ADthub.com/o/r/blob/a1b2c3d/x.py#L12-https://github.com]test_the_round_five_review_counterexamples_are_refusedtest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_the_round_five_review_counterexamples_are_refusedtest_the_round_five_review_counterexamples_are_refusedtest_the_round_five_review_counterexamples_are_refusedtest_the_round_five_review_counterexamples_are_refusedtest_a_trailing_dot_is_dropped_from_a_hosthttps:must be followed by//test_the_round_five_review_counterexamples_are_refused@test_the_round_five_review_counterexamples_are_refusedtest_a_url_of_this_repository_is_refused_however_github_lets_it_be_writtentest_the_round_five_review_counterexamples_are_refused。is not read as.test_a_url_is_read_as_the_url_standard_reads_it[https://github\u3002com/o/r/blob/a1b2c3d/x.py#L12-https://github.com]test_the_round_six_review_counterexample_is_refusedtest_a_long_string_is_scanned_in_linear_timetest_a_long_string_is_scanned_in_linear_timetest_source_names_are_found_where_finditer_finds_themhttp(s):inside a runtest_source_names_are_found_where_finditer_finds_themtest_a_long_string_is_scanned_in_linear_timetest_a_long_string_is_scanned_in_linear_timetest_a_long_string_is_scanned_in_linear_timetest_bindings_are_found_as_a_scan_of_the_whole_string_finds_themtest_bindings_are_found_as_a_scan_of_the_whole_string_finds_themtest_bindings_are_found_as_a_scan_of_the_whole_string_finds_themtest_bindings_are_found_as_a_scan_of_the_whole_string_finds_them,,andor, and, whitespace given back)test_a_long_string_is_scanned_in_linear_timetest_a_long_string_is_scanned_in_linear_timeline(s),l.orll.label is given backtest_a_long_string_is_scanned_in_linear_time\s-test_a_long_string_is_scanned_in_linear_timetest_a_long_string_is_scanned_in_linear_timefinditer)test_a_long_string_is_scanned_in_linear_timetest_a_long_string_is_scanned_in_linear_timex.py- L12)test_the_locators_match_as_before_their_whitespace_was_read_possessivelytest_the_round_seven_review_counterexamples_are_refusedtest_a_string_is_read_as_a_browser_and_as_markdown_delimit_its_urlstest_the_round_seven_review_counterexamples_are_refusedtest_the_round_seven_review_counterexamples_are_refusedtest_the_round_eight_review_counterexamples_are_refusedLnumber after its list's start, not from its lasttest_a_long_string_is_scanned_in_linear_timetest_the_round_eight_review_counterexamples_are_refusedtest_the_round_eight_review_counterexamples_are_refusedtest_the_round_eight_review_counterexamples_are_refusedtest_the_round_eight_review_counterexamples_are_refusedtest_a_string_is_read_again_with_its_markup_escapes_decodedtest_a_string_is_read_again_with_its_markup_escapes_decoded;test_the_round_eight_review_counterexamples_are_refusedtest_the_round_eight_review_counterexamples_are_refusedtest_a_string_is_read_again_with_its_markup_escapes_decodedtest_the_round_eight_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_the_round_eight_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_a_string_is_read_again_with_its_markup_escapes_decodedhtml.unescapedrops ittest_the_round_nine_review_counterexamples_are_refusedtest_a_string_is_read_again_with_its_markup_escapes_decodedtest_a_string_is_read_again_with_its_markup_escapes_decodedtest_a_string_is_read_again_with_its_markup_escapes_decoded=is decodedtest_a_string_is_read_again_with_its_markup_escapes_decodedtest_the_round_nine_review_counterexamples_are_refusedtest_a_string_is_read_again_with_its_markup_escapes_decodedtest_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_a_string_is_read_again_with_its_markup_escapes_decoded=test_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refused..is not resolvedtest_in_repository_paths_resolve_from_an_anchor_or_a_repository_prefix..is skipped, so a path that climbs out resolvestest_in_repository_paths_resolve_from_an_anchor_or_a_repository_prefixtest_the_round_nine_review_counterexamples_are_refusedfa7601cb)test_the_round_nine_review_counterexamples_are_refusedtest_the_round_nine_review_counterexamples_are_refusedtest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_nottest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_not%2etest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_not/or%2f, not\test_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_not.counts as a dot segmenttest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_not../x) is not foundtest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_notdocs/..at a string's end is missed)test_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_nottest_a_line_link_of_this_repository_is_refused_whatever_whitespace_its_query_holdstest_a_line_link_of_this_repository_is_refused_whatever_whitespace_its_query_holdstest_the_round_ten_review_counterexamples_are_refused"test_the_round_ten_review_counterexamples_are_refused'test_the_round_ten_review_counterexamples_are_refusedtest_a_long_string_is_scanned_in_linear_timetest_a_long_string_is_scanned_in_linear_time<test_the_round_nine_review_counterexamples_are_refusedtest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_not<test_the_round_ten_review_counterexamples_are_refusedtest_the_round_ten_review_counterexamples_are_refusedtest_the_round_eleven_review_counterexamples_are_refusedtest_a_line_link_of_this_repository_is_refused_by_its_path_alone%50olicyEngine)test_the_round_eleven_review_counterexamples_are_refusedx\..\in an HTMLhref)test_a_line_link_of_this_repository_is_refused_by_its_path_alonePolicyEngine//microcosm)test_the_round_eleven_review_counterexamples_are_refusedPolicyEngine/x/../microcosm)test_a_line_link_of_this_repository_is_refused_by_its_path_alonetest_a_line_link_of_this_repository_is_refused_by_its_path_aloneNotPolicyEngine/microcosm)test_a_line_link_of_this_repository_is_refused_by_its_path_alonePolicyEngine/microcosm-dynamics)test_a_line_link_of_this_repository_is_refused_by_its_path_alone#readme,#38)test_a_line_link_of_this_repository_is_refused_by_its_path_alonetest_the_round_eleven_review_counterexamples_are_refused%2ftest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_not%2ftest_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_not%2f(..%2fdocs)test_a_dot_segment_refuses_every_line_citation_in_its_string_pinned_or_nottest_a_path_here_that_holds_an_equals_sign_is_read_whole=test_a_path_here_that_holds_an_equals_sign_is_read_whole=test_a_path_here_that_holds_an_equals_sign_is_read_whole%2eas no dot (x/%2E%2e/)test_a_line_link_of_this_repository_is_refused_where_a_dot_segment_forms_its_path%2eas a dottest_a_line_link_of_this_repository_is_refused_where_a_dot_segment_forms_its_path?(LICENSE?a/../../../../..)test_a_line_link_of_this_repository_is_refused_where_a_dot_segment_forms_its_path%2fbefore it resolves dot segments (x%2f..%2fmicrocosm)test_a_line_link_of_this_repository_is_refused_where_a_dot_segment_forms_its_path//x/../../)test_a_line_link_of_this_repository_is_refused_where_a_dot_segment_forms_its_path/([a](…/PolicyEngine/x\y/../microcosm/…))test_a_line_link_of_this_repository_is_refused_where_a_dot_segment_forms_its_patharchived 42ed5d45(round 1's wording)test_no_resource_misspells_a_commit_pin[us]8629dbb, not the register'sa43e8c5test_uk_data_parity_citations_bind_the_incumbent_committest_no_resource_cites_a_line_it_does_not_pin[us]test_no_resource_cites_a_line_in_an_unrecognised_spelling[us]M16 survived the first run: the property's "between" case put the stray sha after the clause break, inside the citation's own clause.
01da9a37adds a placement in the binding commit's clause, and M16 is now killed bycommit 000000a archived 000000a; _.py:1.M27 survived the first run in the clean clone, at
c67030f7. Its earlier kills had come from a counterexample Hypothesis saved in the author's checkout and replayed on every later run. Round 6 adds a deterministic example,x.py; lines 12-19 at a1b2c3d, and clears the database before every run.M78 survived the round-7 run at
ea912094: no generated string put a clause break flush against the citation after it.25b5bbe2adds two examples to the binding differential,at a1b2c3d; on main;x.py:12and9f8e7d6c;x.py:12, and each kills it.M12 survived the round-10 run at
c36671e6, after six rounds that killed it. Its one killer was a 300-example property, and the kill needed a commit in an earlier clause with a;in the generated prose; seeded in-process runs drew that combination in 0 of 100 runs. An explicit example now kills it. Before the round-11 run, each mutant was checked in-process atc36671e6's tests: the mutated helper loaded in place of the helper, no file edited, and each test run on its explicit examples alone (M44, M45, M53 and M54, whose tests are parametrized, only the harness runs). 33 mutants were killed only by random generation: M1, M2, M3, M4, M5, M6, M9, M11, M12, M13, M14, M15, M16, M17, M20, M21, M22, M23, M25, M26, M34, M36, M38, M39, M40, M41, M42, M48, M49, M51, M52, M56, M64. For each, Hypothesis searched for a falsifying input with the mutant loaded, and that input, in readable values, is now an explicit example. Each is shown to pass on the helper and to fail with its mutant applied.Review
Round 1: a GPT-6.1 Sol review was queued with every Codex lane at its limit, and it ran on the Opus fallback (job
20261026-160321-microcosm-1407-review-r1, againstd9672bd9), which had no shell. It returned REQUEST_CHANGES with six findings. Each is now addressed:obr.py:302 on main; compare us-data at 42ed5d4passed); the ESI origin'scps.pywas "pinned" by the unrelated2e044571us-data at 42ed5d4; obr.py:302 on main. The misspelled-pin rule (3) caught the ESI origin, which now spells its own commit. The PUF note now names its commit first. Properties cover each case.x.py:L12,at line,; lines,—,[L12],to/through/andranges, em dash andll.were unrecognisedbuild_loss_matrix.create_target_matrix:93-103stays out of scope, as documented: it is a dotted name, read the same way as ledger keys.github.com/PolicyEngine/microcosm/blob/<sha>/tools/x.py#L12andmicrocosm/tools/x.pyescaped the in-repository ruleutils/loss.py)at e2de2b41…counted as a pin…,...and-end a token rather than a sha, for pins and for misspelled-pin tokensRound 2: GPT-6.1 Sol (job
20261026-184526-microcosm-1407-review-r2, against5221cd6a) ran the guard suite (69 passed; all 18 properties reached their limits with 4,100 passing examples, none vacuous), confirmed each round-1 counterexample is fixed, re-read the round-2 citation edits at42ed5d45, and grepped the 14 superseded hashes. It returned REQUEST_CHANGES with three P2 findings, all fixed inf30da07c:tools/build_us_puf_support_base.py:"12-19", value or key) was no citation, so an in-repository line citation passed#locator now accepts quotes before its number. A new property quotes 8 locator spellings at 4 slots with 8 marks, for foreign and in-repository files, as values and keys.commit a1b2c3d; https://raw.githubusercontent.com/<o>/<r>/main/.../cps.py#L12) or quoted (x.py:12 on `main`) still inherited the earlier commitraw.githubusercontent.com/<o>/<r>/<branch>/,refs/heads/<branch>and quoted branch names. The moving-ref property gains 7 spellings, and a new property cites URLs at 5 moving refs after an earlier commit.look at a1b2c3d/foo.py:12read the directorya1b2c3das a pincommit,ator@, a sha followed by/is no pin; URL pins (/blob/<sha>/x.py) are unchanged. A new property covers hex directories after each prose spelling.All five of the review's inputs are now refused, in a dedicated example test. Re-running the reviewer's own 48 probes at
f30da07cchanges exactly six outcomes, each one of those inputs or a variant of it: the two hex-directory probes,x.py:"12-19"andx.py:"L12-L19"(the second was already refused by the backstop, and is now a citation), and the quoted in-repository value and key. The three fixes change no classification in the live packages: 283 citations, 0 hits under every rule, and all 237 US/UK citations still bind and are in range.The review's "Not verified" generator and coverage checks timed out in its environment (clean-worker import over 300 s); both pass here at
f30da07c, after the merge of90ff43f9: the generator's--checkexits 0, andspec_engine_coverage.py --checkreports 42235/42235 configuration fields and 41/41 inventory checks (106 s, 1.9 GB peak).Round 3: GPT-6.1 Sol (job
20261026-205923-microcosm-1407-review-r3, againstf30da07c) replayed round 2's inputs and probes and confirmed all three are fixed. It ran the guard suite (73 passed; it reports 21 properties at their limits, 4,800 passing examples, none vacuous) and hand-applied M31 and M33, each killed by its intended assertion. It confirmedrepo@<sha>/pathis refused by design, thatmainlandkeeps the earlier-commit fallback, and that a GitLab/-/raw/main/URL is refused. It returned REQUEST_CHANGES with one P2, fixed in1840bc92:https://raw.githubusercontent.com/PolicyEngine/microcosm/refs/heads/main/tools/build_us_puf_support_base.py#L12 at a1b2c3d(and the/blob/refs/heads/main/form) escaped the in-repository rule: the normalizer tookrefsas the ref and leftheads/main/tools/...unresolved/blob/feature/x/tools/..., which the review did not name, had the same gap). Three ref-qualified forms join the in-repository list, a new property cites a file at a ref of 1-4 random segments under 5 URL prefixes, and an example test runs the review's input. No live classification changes: 283 citations, 0 hits, main's counts unchanged.Round 4: GPT-6.1 Sol (job
20261026-215022-microcosm-1407-review-r4, against1840bc92) replayed round 3's input and its/blob/refs/heads/main/form, both now refused. It ran the guard suite (75 passed; 22 properties at their limits, 5,000 passing examples) and hand-applied M35, which both intended tests killed. It exercised dot segments, queries and fragments, ambiguous refs (/blob/tools/tools/...), the neighbouringPolicyEngine/microcosm-dynamics(still foreign) and a file deleted since an old sha (not refused: it resolves against this checkout only, as Scope states). It returned REQUEST_CHANGES with one P2, fixed in4229cd8c:https://github.com/PolicyEngine/microcosm/blob/1840bc92…/%74ools/build_us_puf_support_base.py#L12extracted the file74ools/build_us_puf_support_base.py, foreign and pinned by the URL's sha, and…/tools/build_us_puf_support_base.py?plain=1&review=<32 x's>#L12yielded no citation, because the query was longer than the backstop's 24-character window. Both URLs return HTTP 200%74ools,tools%2F…,…%2Epy,docs/../,docs/%2e%2e/,./, an upper-case host,:443, a user,%62loband the review's query. These return 404:%23L12,%3Fplain=1,/BLOB/andPolicyEngine%2Fmicrocosm. A new property writes a URL of this repository in each way GitHub accepts, another hides line citations behind random queries, and an example test runs the review's two inputs and the forms above. The review's nonblocking finding, an 8,000-segment ref that took 115 s and 1.09 GB to resolve, is fixed as well: the resolver tries only paths no longer than the deepest path form here, and that input now resolves in under a millisecond; a temporary-tree test counts theis_filechecks. URL reading rewrites 69 live strings and changes no classification: 283 citations, 0 hits, main's counts unchanged.The mutant harness's unmutated baseline then caught a generator bug in the new URL property: it could escape a character inside an escape already there, writing
%%32Efor%2E, which names another path.c67030f7fixes the generator and leaves the helper unchanged. Both URL properties then passed under eight Hypothesis seeds, 500 examples each.Round 5: GPT-6.1 Sol (job
20261026-230951-microcosm-1407-review-r5, againstc67030f7) checked round 5 broadly:It returned REQUEST_CHANGES with one P2, fixed in
29983852:https://%67ithub.com/PolicyEngine/microcosm/blob/c67030f7…/tools/ci_test_plan.py#L12extracted67ithub.com/….//github.com:443/…/tools/ci_test_plan.py#L12extracted443/PolicyEngine/…. Both were foreign, pinned by the URL's sha, and served with HTTP 200.//user@github.com/…/%74ools/ci_test_plan.py#L12combined a user with round 4's escape.urlsplitkept the host's escapes, and a GitHub URL without a scheme was read only when a slash followed its hosturlsplit. The host is percent-decoded, mapped to ASCII as UTS46 maps it and lower-cased. A user and a port are dropped,\is/, any number of slashes may lead, and a network-path reference may name any host. Python's IDNA codec was tried first and rejected: its Unicode 3.2 tables readgithub.cₒmas Punycode, where the standard readsgithub.com. The reader was checked against bun'sURLover every code point and 26,350 generated URLs (see Differential tests). An example test runs the review's three inputs, nine more spellings, and four authorities the standard refuses. A table test checks 20 readings recorded from the standard. The URL property gains leads, authorities, full-width and escaped host characters, trailing dots and\separators. Round 6 reads all 4,482,233 live strings exactly as round 5 did, so no classification changes.Round 6: GPT-6.1 Sol (job
20261027-003202-microcosm-1407-review-r6, against29983852) confirmed round 5's three inputs and its probes are refused. Its own differential against bun covered 155,184 URLs, it found 0 reading changes over 4,482,233 live strings, and it ran the guard (102 passed). It hand-applied M53 and M64, both killed. It returned REQUEST_CHANGES with one P2, fixed in5f6647b1:https://github.com/PolicyEngine/microcosm/blob/29983852…/tools/ci_test_plan.py?value='<40 x's>'#L12yielded no citation: the URL token ended at the quote, so#L12was read as prose beyond the backstop's window. Bun reads the quote as part of the query, and GitHub serves the URLtools/'x'/../ci_test_plan.py), which this round found and fixes too. GitHub served each form with HTTP 200, escaped as a browser sends it. An example test runs the review's input with each quote and bracket in a query or a dot segment, and quoted whole URLs. The URL property now writes quotes and brackets in queries and detours, and its indices always land. A code-point differential at four places in the URL agrees with bun everywhere but at whitespace. No live reading or classification changes.The review's nonblocking note, that looking for a user before a GitHub host written without a scheme took time quadratic in a run of punctuation (32,000
:took 1.64 s), is fixed as well: the host is matched where it starts, and a user before it is prose. Hunting for other hostile strings found three older quadratic scans, all fixed here and none new in round 6. Each source name was rescanned from every character of a long run of path characters (60,000atook 19.6 s), and each citation found its string's clause breaks, commits and commit-like tokens again (2,000 pinned citations in one string took 5.8 s). The backstop compared each near match with every citation. Now the source-name scan starts only where a match can start, and each string's marks are found once and searched by bisection. Two differential properties check the new paths againstfinditerand against the whole-string scans, and a timing test runs eight hostile strings in 0.2 s.Round 7: GPT-6.1 Sol (job
20261027-035952-microcosm-1407-review-r7b, againstea912094) confirmed round 6's input is refused. It ran the guard (110 passed; 26 properties, 6,500 passing examples), checked the Unicode, scan and binding differentials and all 4,482,233 live strings, and hand-applied M66, M70 and M75, each killed. It returned REQUEST_CHANGES with two P2s, both fixed in25b5bbe2:[https://example.org/?x](<https://github.com/PolicyEngine/microcosm/blob/29983852…/tools/ci_test_plan.py#L12>)yielded no citation. The URL token ran to whitespace, so the label's query swallowed the link's destination, which CommonMark renders as a link to this repository's file and round 6 had refusedtools/ci_test_plan.py#L12, an HTML<a href>and a compact quoted list. Another pins the two readings: a URL glued to another by a quote is now read as a URL of its own, and a quote in another repository's query is refused in the second reading, the over-refusal Scope states. No live string reads differently the second way.line_citations("L1 and " * 24)took 13.1 CPU seconds and found nothing: a failed list of line numbers was retried once for each way to share its whitespace out,(then an optionaland) orand; and a spaced dash is found by a lookbehind, so it no longer competes with the whitespace before it. A reverse locator is tried once per list, from its head, rather than from each number in it: started inside a list, it reads to the same end, and what ends a reverse locator (of,in) cannot continue a list. Hunting for more hostile shapes found three other superlinear scans, all fixed: a failed locator was retried once for each way to share a run of spaces or quotes out, a reverse locator was tried from each number of a failed list, and a long host label was written in Punycode, whose encoder is quadratic (a label over 63 characters is now left unencoded; see Scope). A differential property checks the new locator patterns against round 7's, and the timing test runs the nine strings below.Each string below, in CPU seconds for
line_citationsandunrecognised_line_locatorstogether, one process per string, stopped at 120 s:ea91209425b5bbe2"L1 and " * 24"lines " + "1 and " * 8 + "in x.py""lines " + "1 and " * 10 + "in x.py""lines " + "1, " * 24 + "in x.py""x.py" + " " * 1_024 + "x""x.py" + "'" * 16_000 + "L1x""line " + " " * 32_768 + "1x in x.py""L1 and " * 8_000Round 8: GPT-6.1 Sol (job
20261027-064542-microcosm-1407-review-r8b, against25b5bbe2) confirmed round 7's two findings are fixed. It replayed 191 round-7 URL probes and 24 boundary additions, and reran round 7's timing input: 14.7 CPU seconds on the old helper, under a millisecond now. It timed 153 hostile workloads at n and 4n and found no superlinear family. Against theea912094locator patterns it compared 377,119 texts and found 0 differences, and against bun it replayed 155,184 URLs. Over all 4,482,233 live strings, every rule's output was unchanged and no string was read a second way. It ran the guard (113 passed; 27 properties, 7,500 examples) and hand-applied three mutants: M84 and M87 were killed, and M80 survived. It returned REQUEST_CHANGES with two P2s, both fixed ina0d20514:L1-L2-3 of tools/ci_test_plan.pyyielded no citation. A reverse locator was tried only from a list's head: it readL1-L2, failed, and moved past the list. But a range can end where another starts, and fromL2the locator readsL2-3and succeedsLnumber, and only then after the list. From any other number it reads the rest of the list as the head did, so only the last number can read differently (see_finditer). Each number is read in at most two lists, so the scan stays linear. The round-8 example test runs the review's input and three more chains. Both differentials gain chain pieces and four examples each. 200,000 random chained lists agree withfinditer[a](https://github.com/PolicyEngine/microcosm/blob/29983852…/tools/ci_test_plan\.py#L12)yielded no citation. CommonMark decodes\.in a link destination, and character references likewise (#L12,#L12,.py);. The URL Standard also deletes ASCII tabs and line breaks from a URL, which reveals a link in anhrefthat decoding alone would not (.../x.p y#L12,ht\ttps://...). So each string is also read with them deleted, as written and as decoded. Every rule checks every reading, and each new reading is added to the old ones, so it can only refuse more (see Scope). The example test runs 15 escaped Markdown and HTML shapes and 4 brokenhrefs. Each was confirmed by markdown-it-py 4.2, or by HTMLParser and bun 1.3. A new property escapes 1-24 characters of a line link of this repository in 6 Markdown and HTML settings (300 examples). Of 200,000 random escapings rendered by markdown-it, 169,582 render to a line link this guard refuses, and all of those are refused as writtenThe review's note on M80 is answered too. The timing test now runs a failed range with 32,768 spaces after its dash (
"lines L1-" + " " * 32_768 + "2x in x.py", and the same withto). M80 takes 30 s on each, against 0.01 s for the helper. The round-8 mutant run found two more survivors, M24 and M81. An explicit worded reverse range (lines 12 to 19 of x.py) now kills M24, and"x.py line" + " " * 65_536 + "1x"kills M81 (13.5 s, against 0.02 s). The review also noted that the second reading refuses a data-source URL whose query quotes a locator-like value (data.json?where=CLASS='L12'). Scope now states that shape.Round 9: GPT-6.1 Sol (job
20261027-085429-microcosm-1407-review-r9, againsta0d20514) confirmed round 8's two findings and the variants it had saved are fixed. 717,075 reverse scans agreed withfinditer, every rule's output was unchanged over all 4,482,233 live strings, and it hand-applied M91, M93 and M100, each killed. Its run of the whole guard file stalled in dependency imports on this host, so it reported none. It returned REQUEST_CHANGES with one P2, fixed infa7601cbandc36671e6:<a href="https://github.com/PolicyEngine/microcosm/blob/29983852…/tools\ci_test_plan.py#L12">yielded no citation of this repository. HTML decodes the reference and leaves the backslash, which the URL Standard reads as/, so the link is totools/ci_test_plan.py. Markdown's one pass took\&as an escaped&, so no reading decoded the reference&or a backslash is now decoded as Markdown decodes it and, apart, as HTML does, each once as text and once as a URL holds what it decodes to, and each is read joined too (see Invariants and Scope). References are now decoded as the HTML standard decodes them. A reference to a control is kept, where Python'shtml.unescapedropped it, so..had read as... The counterexample test runs the review's input and five variants (c, an unquotedhref, leading zeros, a backslash before the path's first segment, and	), each confirmed by parse5 7.3 and bun 1.3. The markup property now writes slashes as\in HTML with a reference after each, and an explicit example puts a decimal reference after a backslash in a double-quotedhrefLooking for the same class, a link a browser reads whole that a reading split, found more, all fixed in round 10. Whitespace in a segment that a later
..removes ended the URL: a quoted attribute value and a destination in angle brackets hold any, and a plain destination and an unquoted value hold a Unicode space or one a reference decodes to.fa7601cbreads enclosed values whole, reads URLs through Unicode spaces, and holds what a reference decodes to inside a URL. A fuzz of that generation against markdown-it, parse5 and bun then found five more links: HTML decodes an attribute's references before the URL Standard strips the value's ends (href=" https://…"), and markdown-it strips whitespace from a destination's ends ([a](< https://… >)).c36671e6strips after decoding. A pinned relative path through an inner..(tools/x/../ci_test_plan.py#L12 at <sha>) was not resolved, and now is. The counterexample test runs 30 inputs, each confirmed by the oracles, and three plain strings these readings must not refuse.Round 10: GPT-6.1 Sol (job
20261027-111834-microcosm-1407-review-r10, againstc36671e6, the delta froma0d20514) lost its lane before it finished (exit 125). Its draft report and evidence were rescued from its checkout. It confirmed round 9's input is refused. It replayed all 45 prior mixed-context variants and all 27,625 prior matrix inputs against fresh markdown-it, parse5 and bun runs: all 8,281 targets are refused, 17 inputs gained a refusal (the backstop's, on\#L12-style non-links) and none lost one. It timed 18 hostile families at 20,000 and 80,000 characters and found none superlinear, and its 396 pinned citations of other repositories gained no refusal againsta0d20514. Its full guard run, live comparison and three mutants were still running when the lane was lost. It reported four P2s. Each is a line link of this repository that markdown-it-py 4.2 renders and parse5 7.3 and bun 1.3 resolve totools/ci_test_plan.py, and in each a..removes a segment that holds whitespace or a backslash. All four are fixed in2822aff1:[a]( <https://github.com/…/tools/a b/../ci_test_plan.py#L12>)yielded no citation of this repository. CommonMark allows spaces, tabs and a line break between(and<, and the enclosure needed](<<, and rule 6 refuses the string as well. The counterexample test writes the gap as a space, a tab, a line break, and spaces and tabs around a line break, and puts a line break before a reference definition's<Use `prefix="` before the value., a blank line, then<a href="https://github.com/…/tools/a b/../ci_test_plan.py#L12">: the code span's="started an enclosure that ended at thehref's opening quote, so no reading held the value[a](<https://github.com/…/tools/a\b/../ci_test_plan.py#L12>): Markdown keeps the backslash and writes it%5C, so the..removesa%5Cb; a reading took the\as/and removedb, citingtools/a/ci_test_plan.py. The plain destination is the same[a]( //github.com/…/tools/a b/../ci_test_plan.py#L12): markdown-it decodes both references, strips the leading space and writes the inner one as%20The review proposed a reading fix for each. Each would close one way a renderer and a reading remove different segments, and every finding needed a
..to remove a segment. So round 11 refuses the class (rule 6): every line citation in a string with a dot segment, in any source its readings start from. Without one, whitespace or a backslash that a reading takes otherwise stays in the segment the renderer follows, and a test checks that no source file here has either in its path. No live string holds a dot segment. The counterexample test runs the four inputs and seven variants, each confirmed by the three oracles. A property puts one of 8 spellings of a dot segment at 7 places beside a pinned or unpinned citation of another repository, and a plain-string test checks that a run of dots that is no whole segment refuses nothing. All 679 of the review's misses are refused now, and its matrix and its over-refusal probes without a dot segment are refused exactly as before (see "How this was verified").Looking for the same class without a
..found one more. GitHub ignores a query, and whitespace in one ended the URL in each reading that no enclosure held (<a href="…/tools/ci_test_plan.py?a b#L12">after round 10's code span), leaving#L12beyond the backstop's window. Where a string holds a?and a#, its URLs are now also read through the whitespace in their query (see Scope). The counterexample test runs 14 query shapes: atc36671e6the 7 an enclosure held were refused, and the other 7 were not. A property writes a line link of this repository with spaces, tabs and Unicode spaces in its query in 8 enclosures.Round 11: GPT-6.1 Sol (job
20261028-014437-microcosm-1407-review-r11, against2822aff1) did not finish. Its first attempt found the model at capacity, and the provider stopped its second after about two hours with a content-filter refusal, so it returned no report. Its session log keeps its running draft: REQUEST_CHANGES with two P2s. Before it stopped, it had hand-applied M130, M138 and M144 (each killed), found no clear superlinear case in its first timing probes, and compared 116,755 live strings (am, be and nz) againstc36671e6with no differences. It also named a candidate it left unconfirmed: GitHub reads%2fas/before it resolves a path. All three are answered in8a62c938:[a](<https://github.com/PolicyEngine/microcosm/blob/feature+foo/tools/ci_test_plan.py#L12>), pinned elsewhere in the string, a reading citesfoo/tools/ci_test_plan.py, which names no file here, so no rule refused it.feature+foois a valid ref namefeature+foo,feature@foo,release!1,v1,2;3=4,it's,a$b,a=b) in Markdown and HTML="has taken the attribute's opening quote:Use `prefix="` before the value., a blank line, then<a href="https://a b@%67ithub.com/PolicyEngine/microcosm/blob/…/tools/ci_test_plan.py#L12">. Every reading's URL ends at the space, and the rest, read as prose, cites67ithub.com/PolicyEngine/microcosm/blob/…/tools/ci_test_plan.py, which names no file here and is pinned by the sha in it.segment%2fbeside..tools/a b%2f..%2fci_test_plan.py#L12renders to a link to line 12 oftools/ci_test_plan.py. Rule 6 now reads%2fas a separator, and rule 7 refuses the link tooIn each finding a renderer follows a link to a line here and no reading does, as in round 10's. So round 12 refuses the class by what the link names rather than adding a reading fix for each: rule 7 needs only this repository's path and a line fragment after it. Its completeness argument and its limits (relative links, redirects, prose before a later fragment) are under rule 7 and Scope. The 22 inputs render as line links of this repository in all three oracles, no rule refused any at
2822aff1, and rule 7 refuses all 22. A test refuses 9 written forms by the path alone and passes 8 near misses, and a property draws refs, authorities, repository spellings, routes, queries and enclosures (see "How this was verified"). No live string is refused.Looking further found one gap of rule 1's own. 18 tracked files hold an
=in their path (docs/evidence/us-import-entry-margins/reconciliation/period=2025-01.json), and a reading starts a name after one. Sodocs/evidence/us-import-entry-margins/reconciliation/period=2025-01.json:3 at a1b2c3dcited2025-01.json, which names no file here, and no rule refused it, andlines 3 ofthat path found no citation. The run of path characters before a cited name is now tried as a path here too, and a reverse locator reads a name after an=. A test covers the forward,microcosm/, reverse and URL forms, and a test that every source path here is read whole checks the premise against this tree.Looking further after round 12 found a gap of rule 7's own. A dot segment can form this repository's path in a link to a file whose name no reading takes for a source:
[a](https://github.com/PolicyEngine/x/../microcosm/blob/<sha>/LICENSE#L3)renders to a link to line 3 ofLICENSEhere, as dox/%2E%2e/,//x/../../,a%2fb/../,x\..\and..in its place, and the same links to the tree'sDockerfileandSHA256SUMS. No reading finds a citation, so rule 6, which refuses citations, never sees one, and rule 7 removed only.segments. Rule 7's completeness argument had counted on rule 6 for dot segments, which holds only where a reading finds a citation. Round 13 (4dc0734e) resolves dot segments in rule 7's own reading, as a browser does before GitHub reads the path. That reading covers every refusal of the one it replaces, which also refused links that a..sends out of this repository. A test refuses 9 such links (round 12's among them) and passes 5 that link no line here (x/../../microcosm,x%2f..%2fmicrocosm,x/..%2fmicrocosm,microcosm-x/../microcosm-dynamics,x/.../microcosm), and the rule-7 property draws three repository paths through a.., two that leave it, the three files and a query of..segments (see "How this was verified").Round 12: GPT-6.1 Sol (job
20261028-061901-microcosm-1407-review-r12b, against8a62c938) found the same gap independently and was cancelled about 25 minutes in, once the fix was pushed. Its draft verdict was REQUEST_CHANGES with one P2, which4dc0734eanswers:.segments, sox/../hid this repository's path, and its completeness argument left parent segments to rule 6, which needs a citation:Use `prefix="` before the value., a blank line, then<a href="https://a b@%67ithub.com/PolicyEngine/x/../microcosm/blob/2822aff1…/tools/ci_test_plan.py?aaaaaaaaaaaaaaaaaaaaaaaaaaaaaa=1#L12">rendered to a link to line 12 oftools/ci_test_plan.py, and no rule refused it4dc0734e), and5578c82dadds this input to the dot-segment test. No rule at8a62c938refuses it; rule 7 refuses it nowBefore it stopped, it had also replayed the 22 round-11 inputs (none refused before, all now), found the new properties non-vacuous, and found that no foreign URL holding one of the 18
=paths is refused by rule 1. Its partial evidence is kept with the review records.A fuzz of rule 7 against the renderers then found 3 links it missed at
4dc0734e, each a Markdown destination holding a backslash (see "How this was verified"). markdown-it percent-encodes a backslash (%5C), so in[a](https://github.com/PolicyEngine/x\y/../microcosm/blob/<sha>/LICENSE#L3)the..removes the segmentx%5Cyand the link is to line 3 ofLICENSEhere, while a browser reading the same URL in anhreftakes the backslash as/. Rule 7 read every backslash as/.a738da7dreads it both ways, and the dot-segment test adds four Markdown shapes (inline, pointy, reference definition, autolink), none refused by any rule at4dc0734e.e5e14e96adds to the rule-7 property a path throughx\y/../and one throughx\..\..\that leaves this repository.Round 13: GPT-6.1 Sol (job
20261028-071421-microcosm-1407-review-r13, against5578c82d) found the Markdown backslash independently and was cancelled about 45 minutes in, once the fix was pushed. Its draft verdict was REQUEST_CHANGES with one P2, whicha738da7danswers:/, but markdown-it percent-encodes one in a destination (%5C), which neither a browser nor GitHub reads as a separator:[a](https://github.com/PolicyEngine/x\y/../microcosm/blob/5578c82d…/LICENSE#L12)rendered to a link to line 12 ofLICENSE, as did its pointy and reference-definition forms, and no rule refused ita738da7dreads the backslash both ways, and the dot-segment test adds an inline link, a pointy destination, a reference definition and an autolink holding one, none refused by any rule at4dc0734e. The review also noted that rule 7's property drew no path through a backslash;e5e14e96adds one that links here and one that leavesBefore it stopped, it had also reviewed all 20 changed paths (each of the 92 changed JSON and YAML leaves is a string, and the four changed production modules' syntax trees are unchanged once string constants are normalized), replayed the 3,960-input matrix (its result equals the author's, and at
8a62c9381,242 of the links passed every rule), rendered a 51,200-input grid of its own with no other miss, reproduced the property space's counts and GitHub's routing, and timed rule 7 on 18 long-input families at n and 4n, with nothing superlinear. Its partial evidence is kept with the review records.Round 14: GPT-6.1 Sol (job
20261028-075749-microcosm-1407-review-r14, againste5e14e96, the whole branch) approved, with no findings. It replayed round 12's and round 13's inputs and the four Markdown shapes, each refused (round 13's family is missed at4dc0734e). It rendered 30,440 inputs of its own, through literal, entity and escaped backslashes, whitespace and controls, Unicode, query and fragment boundaries, distant reference definitions and Markdown and HTML contexts: 653 render to a line link here, and rule 7 refuses all 653. It reproduced the property's 12,800-input space, including the 1,200 refusals that render no line link here. It confirmed that GitHub routesPolicyEngine/x%5Cy/../microcosmandPolicyEngine/..%5Cx/../microcosmto the blob (200) and the encoded-backslash alternatives nowhere (404). It ran the guard tests (137 passed, 31 of 31 properties at their limits), compared every rule over all 200 resources and 4,482,233 strings against8a62c938(no difference; none of the 99 strings naming this repository is refused), and hand-applied three mutants, the backslash read only as/, only as kept, and decoded before dot segments are resolved: each was killed. Its timing found no sustained superlinear trend: split-heavy_browsedsamples took 5.38-5.43 times as long at four times the length, and_links_a_line_here3.67-4.14 times. Its report and evidence are kept with the review records.New Zealand
The NZ files stay as #1282 left them. This guard scans the NZ package too and finds nothing under any of the seven rules: every NZ line citation is covered by a full-path
path/commitpin. The NZ hub agreed to keep NZ in this guard's scope and not to fold #1282's grammar into the shared helper: that grammar accepts any file extension and would over-match US and UK prose. The guard's docstring says to ask the NZ hub before exempting any NZ resource that starts failing it.axiom: n/a: spec citation text, test-only validators and re-pins; no policy or rule change.
🤖 Generated with Claude Code