Skip to content

Add macOS assigned-process-group containment for Watcher detectors - #257

Merged
lossyrob merged 6 commits into
mainfrom
claude/telex-issue-172-586bf6
Nov 15, 2026
Merged

lossyrob merged 6 commits into
mainfrom
claude/telex-issue-172-586bf6

Conversation

@lossyrob

@lossyrob lossyrob commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Closes #172

macOS source builds now run trusted non-detaching Watcher detectors and send events through the identity-verified InstalledCurrent Rust client. This branch incorporates main through #260, including the macOS daemon bootstrap correction; sender readiness and SQLite/PostgreSQL lifecycle and health coverage are enabled here.

Implementation

  • Each attempt uses a private guard outside the assigned detector group. The detector starts only after the runtime durably records boot, guard, leader, PGID, nonce and receipt identity.
  • Cleanup retains the unreaped leader through every nonzero group signal. A fresh terminal wait plus an unsaturated two-slot census must show only the leader before signal authority is sealed, the exact leader is reaped, and ESRCH-only group absence permits the durable receipt.
  • The guard watches runtime death through kqueue independently of control-handle EOF. A leaked handle cannot keep detectors running after runtime death. Descriptor cleanup enumerates the actual open descriptors instead of imposing a numeric ceiling.
  • Admission checks native architecture, translation, kernel configuration and required capabilities. OS versions are diagnostic/coverage information, never an admission allowlist.
  • The installation tool supports macOS permission checks and pins the hash of the matching adjacent process guard. The existing runtime, resilience, hardening and installation suites now execute on macOS.
  • CI covers macos-15, macos-26 and macos-15-intel, including isolated live PostgreSQL and SQLite sender/lifecycle/health checks. Guardian staging runs its process probe after staging, supporting Cargo's explicit test guard handoff and relocated test executables.
  • Diagnostic subprocess tests use an isolated environment so unrelated runner credentials do not alter their expected error codes.

Validation

Native Apple Silicon Mac: macOS 26.7.1 / 25G241, SDK 26.5.

  • Watcher library: 98 passed, 2 subprocess fixtures ignored.
  • Explicit live PostgreSQL hardening, backend outage, sender outage and runtime matrix: 4 passed, with TELEX_PG_REQUIRE=1. The hardening battery exercises 100 watches and crash/replacement recovery.
  • CLI: 30 passed; installation: 13 passed; process/native conformance: 25 passed; diagnostics: 3 passed. Ignored entries are subprocess roles and the separately hosted PID/PGID reuse stress test.
  • Guardian staging tests: 5 passed; exact Cargo guardian staging/process probe passed.
  • Workspace formatting and Watcher all-target Clippy with warnings denied passed.

Hosted validation at exact head 4d57063e5a5593f89010c471b11042eb571b9ab7 is green: all 26 CI jobs and all 3 Windows archive jobs passed.

Native runner Actual OS Architecture
macos-15 macOS 15.8.2 arm64
macos-26 macOS 26.7.2 arm64
macos-15-intel macOS 15.7.9 x86_64

Each hosted native job passed 10 containment unit checks, 25 process/conformance tests, the separately executed PID/PGID reuse stress test, 98 library tests including live PostgreSQL, 30 CLI tests, 13 installation tests and 3 diagnostic tests. PostgreSQL was required and executed; these are not skipped-backend passes. The actual runner set records coverage, not runtime admission policy. The prior Linux guardian-staging failure is fixed.

Limits

Detectors must remain in the assigned process group. ADR 0057's accepted transient-invisibility residual remains: Ended means the recorded predicate held, rather than a stronger descendant-containment claim. A leaked stdout/stderr pipe end can misclassify a conforming detector after the bounded drain; a leaked stdin write end can defer EOF until timeout. Runtime death is independently observed despite those leaks.

Release-archive packaging/distribution and production soak remain separate. This PR adds no privileged or persistent helper, entitlement, public guard endpoint, or version-number admission gate.

Implement the macOS guard for trusted non-detaching detectors (#172,
ADR 0054 as amended by ADR 0057).

- A private per-attempt guard runs in its own session and forks a gated
  trampoline that becomes the process-group leader. The detector execs
  only after the runtime durably records the full identity (boot-session
  UUID, guard and leader PID and start time, PGID, nonce, receipt path).
- The guard keeps the leader unreaped through every nonzero group signal.
  Completion needs a fresh nonreaping waitid plus a two-slot
  proc_listpids census showing only the leader. Signal authority is then
  consumed before the exact reap, and only kill(-PGID, 0) == ESRCH lets
  the guard write and sync the Ended receipt.
- Admission is capability and safety checks only (native arm64/x86_64,
  not Rosetta-translated, release kernel, probes on the gated leader).
  The OS version is never an admission input.
- The guard restores default SIGCHLD handling and closes inherited
  descriptors at startup. An escaped process still holding the output
  pipes after the group ended fails the attempt after a bounded drain.
- Enable the existing process tests on macOS and add a native
  conformance suite, fixture modes and a hosted macOS CI job.
- Describe the capability in the runtime and design docs.

The runtime matrix tests stay disabled on macOS: the bootstrapped daemon
connection used by senders is not supported there yet.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@lossyrob

Copy link
Copy Markdown
Owner Author

Design steward (Watcher workstream): acceptance boundary for this PR. Implementation and design review at the exact head will follow in a separate comment.

This PR is the single #172 product PR, so Closes #172 stays. #172 closes only when this PR merges with all three acceptance items in the #172 body met at the merged head. Reviewers do not have an option to close #172 with any of them open, and #172's scope will not be quietly reduced.

  1. Native conformance passes on the operator's Mac and on hosted macos-15 / macos-26 arm64 runners, plus x64 where available. At df1437a, all three Watcher containment conformance legs fail.
  2. Independent implementation and design review passes at the exact head. The steward is running this review now; there is no separate reviewer assignment.
  3. Full CI and SQLite/PostgreSQL lifecycle and health parity pass, without weakening the receipt, sender, store-binding or readiness fences. You disclose that macOS senders never become ready (ensure_bootstrap_mapped_identity_supported in src/daemon.rs) and that the runtime-matrix and installation.rs tests stay disabled on macOS. Item 3 is therefore not met: the Mac parity tests must be enabled and must pass.

Answer to the question in the body: #172 cannot close while macOS senders are unready. The daemon's mapped bootstrap identity on macOS is a real prerequisite outside containment. It is being classified with the Local daemon lane. It will either be absorbed here or land as its own upstream item, with #172 depending on it. Either way, this PR becomes acceptable only once senders are ready and parity passes at its head.

Other points from the body:

Acceptance is not changed: no kernel, finite-run or exact-build gate returns, and no per-run grant is needed for ordinary CI. Campaign runs the merge gate.

@lossyrob

Copy link
Copy Markdown
Owner Author

Addendum to the steward boundary above, at df1437a: the ApplicationClient consumer disposition for this PR is UNSATISFIED.

Before an effective Closes #172 can land, the same head needs:

  • a runnable macOS InstalledCurrent route;
  • sender readiness;
  • SQLite/PostgreSQL lifecycle and health parity.

Disabled tests and containment-only evidence do not substitute. This is the same requirement as acceptance item 3, now also confirmed by the consumer. Local daemon ownership of the mapped-image/bootstrap prerequisite is unchanged.

@lossyrob

Copy link
Copy Markdown
Owner Author

Design steward: implementation and design review of PR #257 at exact head df1437a (base b0d63db). Verdict: CHANGES REQUIRED.

This covers acceptance item 2 for the #172 product PR, reviewed against operator decision 6830371292 as amended by 6835298564, ADR 0054 as amended by ADR 0057, and the #172 body. It is not a merge approval; Campaign runs the merge gate.

Method: steward design pass plus one independent static implementation review, both read-only at this exact head (diff, head files, hosted CI logs). Nothing was built or run. XNU semantics are from documentation and memory, not a new kernel audit.

Containment core: conforms

No violation of the containment invariants was found.

  • Census: int[2] PROC_PGRP_ONLY. Only a 4-byte result naming the leader passes. Errors, zero, malformed, saturated and mismatched results all fail.
  • Terminal check: a fresh zeroed waitid(P_PID, WEXITED|WNOHANG|WNOWAIT) on every attempt.
  • Seal: the terminal-plus-census pair is taken twice. Anchor -> Sealed -> FormerGroup; nonzero kill exists only on Anchor (kill_group and Drop). The leader is reaped exactly, with a status consistency check.
  • Absence: kill(-PGID,0). Only ESRCH is absent; 0 and EPERM are present. The receipt is written only after this, with create_new, sync_all, a link and a directory fsync.
  • Recovery: clears only on an exact identity-matched finalized receipt or a changed kern.bootsessionuuid, otherwise Unproven. No numeric PID is ever signalled.
  • Admission:
    • proc_translated ENOENT and 0 are native; other values and errors are refused.
    • Hardware and process architecture are cross-checked.
    • Non-RELEASE_ kernels are refused.
    • The OS version is never an input.
    • The guard repeats admission and probes the gated leader.
    • Refusal produces no receipt.
  • Guard startup restores default SIGCHLD.
  • Linux and Windows change only in cfg lists and the new MacosGuard variant.

Blocking

B1. Native conformance has produced no evidence.

  • All three Watcher containment conformance legs (macos-15, macos-26, macos-15-intel) fail with start process guard: No such file or directory.
  • Cause: process_macos.rs:476-481 finds the guard beside current_exe(), dropping a trailing deps directory. Hosted Cargo now runs test binaries from target/debug/build/telex-watcher/<hash>/out/, so the guard is not there.
  • This fails closed, since no detector ran, but it means acceptance item 1 has no hosted evidence.
  • Fix: give tests an override fed from CARGO_BIN_EXE_watcher_process_guard as a test-only crate API, never an environment variable honoured in production. Or resolve the new layout. Do not change how production finds the guard beside the installed executable.
  • The macos-26 label does exist; that leg ran.
  • The Ubuntu and Postgres failures (tests/support/telex_isolation.rs:56, "branch telex binary not found") come from the same layout change and also fail on main at b0d63db. They are not caused by this PR. Full CI must still be green at the head that is accepted.

B2. A leaked control descriptor can disable runtime-death cleanup.

  • macOS has no SOCK_CLOEXEC or pipe2. UnixStream::pair() (process_macos.rs:468) and std's stdio pipes therefore set close-on-exec in a separate step, and std's posix_spawn does not use POSIX_SPAWN_CLOEXEC_DEFAULT.
  • So a non-guard child that the runtime spawns at that moment can inherit the runtime end of control.
  • If the runtime is then SIGKILLed or crashes, the guard never sees EOF. It has no independent timeout, so the detector runs unbounded until it exits by itself.
  • That breaks two [telex-watcher] Implement and prove macOS assigned-group containment #172 outcomes: private runtime-liveness observation, and terminating the group on every cleanup path. Abrupt runtime death is one of those paths.
  • This is not ADR 0057's accepted residual, so it cannot stay as a disclosure.
  • Required: the guard must observe runtime death independently of descriptor EOF. For example:
    1. pass the runtime PID;
    2. check that getppid() matches and is not 1;
    3. register kqueue EVFILT_PROC/NOTE_EXIT on it, where ESRCH means the runtime is already dead;
    4. re-check getppid();
    5. treat runtime exit like EOF in the release and wait loops.
  • Add a conformance case: runtime SIGKILL while a third-party process holds the control descriptor, and the group still Ends.

B3. Docs must not move sender readiness out of #172.

Acceptance prerequisite (steward boundary and consumer addendum above)

Non-blocking (fix, or record as a known limitation in docs/watcher-runtime.md)

  • N1. If a leaked stdout or stderr write end is held by an unrelated process, a conforming detector hits the 2 s drain and is labelled nonconforming. Containment correctly stays Ended. If the detector inherits the guard's stdin, a leaked stdin write end can also hang an EOF-reading detector until its timeout. Moving pipe creation into the guard (passing the runtime ends back with SCM_RIGHTS) would fix both.
  • N2. descriptor_limit() clamps at 65,536 (process_macos.rs:799-801). Descriptors numbered above that survive the guard sweep and the trampoline close loop. Enumerate the open descriptors (PROC_PIDLISTFDS), or use rlim_cur and refuse RLIM_INFINITY.
  • After docs: remove macOS version-number admission gates #255 (Docs: record macOS Watcher no-version-gating admission policy #254) merges, rebase. ADR 0057's admission text is owned there; do not edit it here.

Send a new exact head here once these are addressed. The re-review will cover only the changed parts and the acceptance gates, not the core again.

@lossyrob lossyrob left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGES REQUIRED at df1437a.

Publishing the existing complete steward review from #257 (comment) as a submitted PR review. The findings below retain their original exact-head scope and static-review limitations; this is not a new audit or native execution. GitHub does not permit a formal request-changes review on one's own PR, so this is submitted as COMMENT; the technical verdict remains CHANGES REQUIRED, with no acceptance or merge approval.

Integration correction: the original final instruction to "rebase" is superseded by Campaign's ordinary-merge/no-rebase policy. After #255 merges, integrate current main with an ordinary merge while preserving its owned ADR 0057 changes. The CI discussion below describes the original reviewed head, not a fresh claim about upstream #258. The operator is handling the separate #259 external Mac-worker handoff; no implementation assignment is inferred here.


Design steward: implementation and design review of PR #257 at exact head df1437a (base b0d63db). Verdict: CHANGES REQUIRED.

This covers acceptance item 2 for the #172 product PR, reviewed against operator decision 6830371292 as amended by 6835298564, ADR 0054 as amended by ADR 0057, and the #172 body. It is not a merge approval; Campaign runs the merge gate.

Method: steward design pass plus one independent static implementation review, both read-only at this exact head (diff, head files, hosted CI logs). Nothing was built or run. XNU semantics are from documentation and memory, not a new kernel audit.

Containment core: conforms

No violation of the containment invariants was found.

  • Census: int[2] PROC_PGRP_ONLY. Only a 4-byte result naming the leader passes. Errors, zero, malformed, saturated and mismatched results all fail.
  • Terminal check: a fresh zeroed waitid(P_PID, WEXITED|WNOHANG|WNOWAIT) on every attempt.
  • Seal: the terminal-plus-census pair is taken twice. Anchor -> Sealed -> FormerGroup; nonzero kill exists only on Anchor (kill_group and Drop). The leader is reaped exactly, with a status consistency check.
  • Absence: kill(-PGID,0). Only ESRCH is absent; 0 and EPERM are present. The receipt is written only after this, with create_new, sync_all, a link and a directory fsync.
  • Recovery: clears only on an exact identity-matched finalized receipt or a changed kern.bootsessionuuid, otherwise Unproven. No numeric PID is ever signalled.
  • Admission:
    • proc_translated ENOENT and 0 are native; other values and errors are refused.
    • Hardware and process architecture are cross-checked.
    • Non-RELEASE_ kernels are refused.
    • The OS version is never an input.
    • The guard repeats admission and probes the gated leader.
    • Refusal produces no receipt.
  • Guard startup restores default SIGCHLD.
  • Linux and Windows change only in cfg lists and the new MacosGuard variant.

Blocking

B1. Native conformance has produced no evidence.

  • All three Watcher containment conformance legs (macos-15, macos-26, macos-15-intel) fail with start process guard: No such file or directory.
  • Cause: process_macos.rs:476-481 finds the guard beside current_exe(), dropping a trailing deps directory. Hosted Cargo now runs test binaries from target/debug/build/telex-watcher/<hash>/out/, so the guard is not there.
  • This fails closed, since no detector ran, but it means acceptance item 1 has no hosted evidence.
  • Fix: give tests an override fed from CARGO_BIN_EXE_watcher_process_guard as a test-only crate API, never an environment variable honoured in production. Or resolve the new layout. Do not change how production finds the guard beside the installed executable.
  • The macos-26 label does exist; that leg ran.
  • The Ubuntu and Postgres failures (tests/support/telex_isolation.rs:56, "branch telex binary not found") come from the same layout change and also fail on main at b0d63db. They are not caused by this PR. Full CI must still be green at the head that is accepted.

B2. A leaked control descriptor can disable runtime-death cleanup.

  • macOS has no SOCK_CLOEXEC or pipe2. UnixStream::pair() (process_macos.rs:468) and std's stdio pipes therefore set close-on-exec in a separate step, and std's posix_spawn does not use POSIX_SPAWN_CLOEXEC_DEFAULT.
  • So a non-guard child that the runtime spawns at that moment can inherit the runtime end of control.
  • If the runtime is then SIGKILLed or crashes, the guard never sees EOF. It has no independent timeout, so the detector runs unbounded until it exits by itself.
  • That breaks two #172 outcomes: private runtime-liveness observation, and terminating the group on every cleanup path. Abrupt runtime death is one of those paths.
  • This is not ADR 0057's accepted residual, so it cannot stay as a disclosure.
  • Required: the guard must observe runtime death independently of descriptor EOF. For example:
    1. pass the runtime PID;
    2. check that getppid() matches and is not 1;
    3. register kqueue EVFILT_PROC/NOTE_EXIT on it, where ESRCH means the runtime is already dead;
    4. re-check getppid();
    5. treat runtime exit like EOF in the release and wait loops.
  • Add a conformance case: runtime SIGKILL while a third-party process holds the control descriptor, and the group still Ends.

B3. Docs must not move sender readiness out of #172.

  • docs/design/watcher.md (~L733-736) says macOS is unshipped "until its packaging and sender path are accepted separately".
  • docs/watcher-runtime.md says the bootstrapped daemon connection "is not yet supported there".
  • Sender readiness and SQLite/PostgreSQL parity are #172 acceptance item 3, not a separate acceptance. Release-archive packaging may remain separate.
  • At the head that closes #172, these sentences must describe a working sender path and qualify only packaging and distribution.
  • Until then, keep the #177 fail-closed sentence. The #172 body allows it to be removed only when the complete capability lands at that exact head.

Acceptance prerequisite (steward boundary and consumer addendum above)

  • The sender path depends on upstream #259 (#259): macOS daemon/bootstrap mapped-identity support. Campaign has admitted it as a bounded Local item with #172 depending on it. Its owner is being arranged separately, not assigned to this PR.
  • Once #259 lands, this PR needs, at its own head:
    • a runnable macOS InstalledCurrent route;
    • legitimate sender readiness;
    • the macOS runtime-matrix and installation.rs lifecycle and health parity tests enabled and passing;
    • full CI green.
  • Closes #172 stays. The PR cannot be accepted before then, and no reviewer option narrows that.

Non-blocking (fix, or record as a known limitation in docs/watcher-runtime.md)

  • N1. If a leaked stdout or stderr write end is held by an unrelated process, a conforming detector hits the 2 s drain and is labelled nonconforming. Containment correctly stays Ended. If the detector inherits the guard's stdin, a leaked stdin write end can also hang an EOF-reading detector until its timeout. Moving pipe creation into the guard (passing the runtime ends back with SCM_RIGHTS) would fix both.
  • N2. descriptor_limit() clamps at 65,536 (process_macos.rs:799-801). Descriptors numbered above that survive the guard sweep and the trampoline close loop. Enumerate the open descriptors (PROC_PIDLISTFDS), or use rlim_cur and refuse RLIM_INFINITY.
  • After #255 (#254) merges, rebase. ADR 0057's admission text is owned there; do not edit it here.

Send a new exact head here once these are addressed. The re-review will cover only the changed parts and the acceptance gates, not the core again.

lossyrob and others added 5 commits November 15, 2026 00:16
- Tests name the guard Cargo built through a test-only crate API; the
  installed runtime still finds the guard beside its own executable.
- The guard watches the runtime process with kqueue NOTE_EXIT, so runtime
  death ends the attempt even when another process holds a leaked copy of
  the control endpoint. Adds a conformance case for that.
- Descriptor sweeps enumerate open descriptors (PROC_PIDLISTFDS) instead
  of a clamped numeric range.
- Docs keep the macOS fail-closed text until #172 lands with sender
  readiness and parity (#259), and record leaked pipe ends as a known
  limitation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…pdate

# Conflicts:
#	.github/workflows/ci.yml
@lossyrob
lossyrob merged commit 091942b into main Nov 15, 2026
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[telex-watcher] Implement and prove macOS assigned-group containment

1 participant