Skip to content

[test-sentinel] Resolve branch Telex binary from relocated Watcher test executables #256

Closed

Description

@lossyrob

Problem

Docs PR #255 is blocked by a shared process-fixture setup failure in three jobs of CI run 43306999770, attempt 1. The source head is eb6b802813ec52b2f51bcdc213e9521a2ff08f40, base b0d63dbb794e40e1f130f50b2243db9fcb1a7aed; the provider merge 9d621782a1ff3f332c224ccbb08d4e17b791fbae has that same source tree.

The Windows InstalledCurrent Postgres, Ubuntu build/test and live Postgres jobs each report four applicable hardening/resilience/outage/matrix failures before runtime assertions. tests/support/telex_isolation.rs:56 reports:

branch telex binary not found at <target>/debug/build/telex-watcher/<hash>/out/telex[.exe]

The running Watcher test executable is beneath that out directory. The shared branch_binary() fallback only walks upward when the executable's parent is named deps; otherwise it treats the parent as the profile directory. Its compile-time CARGO_BIN_EXE_telex candidate was absent or unusable in these runs; the logs do not distinguish which.

Full tracker-first evidence, job links, log hashes and counts: health log receipt. The resolver blob 41359c65e78073f2196e03acd3d05f8b69d72057 is identical at base and head. All three jobs also updated stable Rust from 1.99.1 to 1.100.0, but that is a co-observation, not a proven toolchain cause. The separate macOS station-intent cap-setup occurrence remains its own classification.

Required outcome

Restore deterministic resolution of the exact branch-built Telex executable for the affected Watcher fixtures, including the observed relocated executable layout. Investigate the existing build/fixture handoff and reuse existing resolver or artifact-identity mechanisms where appropriate; do not assume that directory walking alone is the correct repair.

Keep the process fixtures isolated and bound to the current branch build. Never fall back to PATH, a user's installed Telex, a released binary, or an arbitrary stale executable. Missing or ambiguous required artifacts must produce an explicit setup failure.

Scope and acceptance

  • Make a minimal test/build-scaffolding correction; preserve runtime contracts, public APIs, features, selectors, assertions, timeouts and backend coverage.
  • Add focused coverage for the broken resolution/handoff and missing-artifact behavior, including the relevant ordinary and relocated test-executable paths.
  • Exercise the affected SQLite and Postgres process-fixture cases with the actual candidate, and obtain corresponding Windows InstalledCurrent Postgres, Ubuntu build/test and live Postgres CI evidence.
  • Preserve exact build identity and strict manifest matching. Do not hide the failure through a toolchain downgrade, skipped tests, relaxed assertions or a blind green rerun.
  • Record the actual cause and evidence limits. Rust-version causality must not be claimed without a controlled comparison.

Ownership and delivery

Campaign routes this narrow shared Application Client/InstalledCurrent fixture repair through the existing Application Client coordinator. Keep one implementation owner, normal independent review and its own closing change; do not mix the repair into the six-path macOS policy docs PR.

This is upstream CI-unblocking work for #255 / #254, not a dependency or new acceptance gate for Mac Watcher product #172. Keep #172 ownership and geometry, #252 completion and the operator's no-version-gating policy unchanged. No product, release, shared installation or host action is authorized by this test repair.

Related health log: #182.

Activity

  1. lossyrob commented on Nov 12, 2026

    @lossyrob
    OwnerAuthor

    The original CI attempt is now terminal: 22 jobs, 17 successful and 5 failed. A fourth job independently hit the same binary-resolution setup frontier: Windows Build and test, job 131492812306, with four SQLite fixture failures at telex_isolation.rs:56 and the test executable under build/telex-watcher/efcc4a720278c8de/out.

    Tracker-first classification and raw evidence: #182 (comment) . Its module result is 78 passed, 4 failed and 2 ignored; 143.61 seconds is suite duration, not a measured resolution or lock-acquisition interval.

    Thus four jobs show this shared setup boundary; the fifth failed job is the separately recorded macOS station-intent cap-setup occurrence. This does not establish a Rust-version cause or justify blindly retrying the resolver failures.

    Campaign is routing the narrow repair through the existing Application Client coordinator. The docs PR remains held pending the separately reviewed correction and fresh integrated CI; no Mac Watcher product gate is added.

  2. lossyrob commented on Nov 12, 2026

    @lossyrob
    OwnerAuthor

    Owner: session 818c549c-a544-4d97-8542-d7d6ca5c5fb9

    I will repair the shared branch-binary fixture resolver, cover ordinary and relocated Cargo layouts plus missing and ambiguous artifacts, and preserve strict build identity and process isolation. Source research found the existing compile-time Cargo binary handoff and build-id mechanism; the current CI build steps do not persist a separate artifact handoff.

    The candidate will receive independent exact-head review and Windows/Linux SQLite/Postgres CI evidence. Product behavior and the separate documentation and macOS policy work remain with their current owners. The recorded logs do not distinguish absent from unusable CARGO_BIN_EXE_telex, and Rust 1.100 is co-observed rather than a proven cause.

  3. lossyrob commented on Nov 13, 2026

    @lossyrob
    OwnerAuthor

    Field report: #256

    The branch-built fixture repair is merged and validated on main. PR #258 merged at 2026-11-13T00:05:00Z; #256 is CLOSED/COMPLETED. Both exact-merge workflows finished successfully on their first attempts: main CI, 22/22 jobs; archive workflow, 3/3 jobs. No rerun was performed.

    Merge and reviewed-content binding

    • Merge: 6456d828a18105c6fc9991cca1220a5318eb7363
    • First parent/main: 3bb75ce397a58f790429bee5b348d1ea0c9106a1
    • Second parent/reviewed candidate: 7eb562f07cabebe914e639ec99b2abba718fb8a2
    • Merge tree: 2c1c46cfa823f86aaa5bec3974bb7e055339e53b

    GitHub commit/tree metadata confirms this ordered ancestry. The six load-bearing merge blobs match the independently reviewed candidate exactly:

    Repository path Git blob
    tests/support/telex_isolation.rs 3d9082a0286edacb75ffbdb47ff03045474e9070
    telex-watcher/tests/cli_v2.rs 18cc31f9a04b19d41dc35b9ae9705a0b6ee534b6
    tests/stage_watcher_test_guard.py c3786938a4e520243309e5ca3438e81d1ea05630
    tests/stage_watcher_test_guard_test.py 081abab30525aeedfb6be81f41542ad7c85f0608
    .github/workflows/ci.yml 1bfa2f2254a596f5c917331d51366684bf737c46
    docs/application-client-core.md 07f890e2c21bb6c5bab1014b5030fcfbe2e6cf43

    What held and what changed

    The shared fixture preserves an existing compile-time CARGO_BIN_EXE_telex as its preferred candidate. Its fallback is bounded to ordinary Cargo deps and the observed relocated Watcher layout, canonicalizes candidates, and rejects missing or ambiguous artifacts explicitly. Every selected executable must pass an isolated, bounded exact linked-build-ID probe. The Windows CLI fixture now reuses that resolver.

    Linux test preparation selects the guardian and emitted test hosts from successful Cargo JSON records bound to the exact Watcher manifest. It stages the guardian where unchanged production lookup expects it. Workspace, live Postgres and late all-features capacity preparation each cover their own emitted hosts. Production process lookup and detector logic are unchanged; strict identity/manifests, isolated roots, assertions, selectors, features, timeouts and backend coverage are preserved.

    Successors must repeat the artifact handoff when a later feature set or selector emits another test host. The existing build ID is not a feature fingerprint; exact build identity and matching-feature preparation are separate requirements. Design impact: updated-docs.

    Exact-merge hosted evidence

    Both runs are push executions on main, head_sha=6456d828a18105c6fc9991cca1220a5318eb7363, run_attempt=1, terminal success. Every listed job is terminal success at that same SHA.

    Main CI run 43334164102, 22/22:

    Job Result
    131586001751 - Build and test (ubuntu-latest) success
    131586001801 - Build and test (windows-latest) success
    131586001578 - Live Postgres parity tests success
    131586001571 - InstalledCurrent Postgres process proofs (Windows) success
    131586001545 - Copilot fallback E2E (macos-latest) success
    131586001741 - Copilot fallback E2E (windows-latest) success
    131586001690 - Credential invocation lifecycle (macOS) success
    131586001990 - Feature combinations build success
    131586001135 - Released v0.2.1 upgrade proof (Windows) success
    131586001792 - Station-intent lock boundary (macos-latest) success
    131586001831 - Station-intent lock boundary (ubuntu-latest) success
    131586001572 - Station-intent lock boundary (windows-latest) success
    131586001849 - Watcher runtime diagnostics (macOS) success
    131586001667 - Windows heap diagnostics success
    131586001697 - Windows token-user alignment (all) success
    131586001832 - Windows token-user alignment (default) success
    131586001615 - Windows token-user alignment (entra) success
    131586001969 - Windows token-user alignment (postgres) success
    131586001871 - Windows token-user alignment (sqlite-entra) success
    131586001738 - Windows token-user alignment (sqlite-postgres) success
    131586001941 - Windows token-user alignment (sqlite-self-update) success
    131586001670 - Windows token-user alignment (sqlite) success

    Archive run 43334164085, 3/3:

    Job Result
    131586001456 - Watcher archive fixture contracts (ubuntu-latest) success
    131586001397 - Watcher archive fixture contracts (windows-latest) success
    131586001309 - Windows x64 archive install and genuine source upgrade success

    The retained logs confirm all four SQLite Watcher batteries and the CLI stall-detector assertion passed on Ubuntu and Windows. Ubuntu's late all-features step staged the guardian beside its emitted 2c8b3de22586177b/out host, then passed the original hardening assertion in 104.53s. Linux Postgres passed all four Watcher batteries in 349.84s and 32 InstalledCurrent proofs; Windows Postgres passed all four Watcher batteries in 367.06s. Archive packaging, installation and receipt-preserving genuine-source upgrade steps passed in their hosted job.

    Independent review and evidence limits

    Fresh independent review session 25e371bf-96e5-4b8f-bbc4-b2e864a2adb0 / App identity 58a3e6b4-f765-4f78-a25a-af763cdb9ee5, configured PAW-Review / gpt-6.1-sol / xhigh / long_context / interactive, returned APPROVE on candidate 7eb562f07cabebe914e639ec99b2abba718fb8a2 against base 3bb75ce397a58f790429bee5b348d1ea0c9106a1: zero blocking or actionable non-blocking findings. It explicitly found strict branch-build identity and fail-closed behavior preserved.

    Methods included the entire six-file diff and unchanged production lookup, five Rust resolver tests, five Python staging tests plus supplemental probes, locked builds, exact Windows CLI/process fixtures, all four Windows SQLite batteries, manual relocated Windows runtime matrix, and workflow syntax/static feature-selector-budget checks. This was an independent local implementation verdict, not a submitted GitHub approval or a review performed on the merge commit. The reviewer did not independently execute Linux guardian, local live Postgres, full workspace or hosted CI; exact-merge hosted observations above are separate evidence.

    The investigation remains bound to receipt 6837694688, receipt 6837722764, and terminal predecessor receipt 6838573068. Original tracker context: 6836237919 and 6836340965. The bounded successor explanation is PR receipt 6838753555; the final candidate/review/CI packet is tracker receipt 6839557034.

    Original logs do not distinguish absent from unusable CARGO_BIN_EXE_telex. Rust 1.100 is co-observed, not a proven cause. A controlled pre-staging probe confirms the missing out/watcher_process_guard path and ENOENT; the original generic detector failures and later capacity failure did not themselves expose the underlying spawn errno.

    #167 remains OPEN/reopened. Its unchanged Windows delayed-writer test passed in this merge run, but that non-recurrence neither selects its cause nor completes its separate investigation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions