Repository navigation
fix(aw-sync): reconcile owner-originated event deletions - #855
TimeToBuildBob wants to merge 13 commits into
Conversation
A source-side delete inside the 7-day reconcile window leaves the destination row in place forever: reconcile_updated_events only walks identities that still exist on the source, and the incremental copy resumes at the destination's newest event end. Both propagation tests fail on 10b4b7f. The two contract tests in the same file (7-day limitation, empty-source-window guard) pass before and after. Git-Session-Id: d15e1888-bd37-4cc5-b7a1-8faf30bf1162
`reconcile_updated_events` only walks identities that still exist on the source, so a deleted event had no representation and its destination row stayed forever. Under the single-writer model the source is authoritative for its own bucket — the same argument that makes a differing payload an edit rather than a conflict — so absence inside the same bounded window is enough: no tombstone record is needed. After the edit pass, any remaining destination-only identity is removed unless the source still backs it. Two guards keep that from destroying data: - a row whose timestamp still exists on the source is an edit the incremental path owns (e.g. a shrunken latest-event duration moves the identity off the resume boundary), and a row whose clip-invariant (end, data) fingerprint — the resume-boundary dedup scheme — still exists on the source is a start-clipped copy of a live event; - an empty source window is indistinguishable from a reset datastore, so it never prunes. `reconcile_updated_events` now returns the number of rows it removed, so `sync_one` can keep the ActivityWatch#682 "count must not shrink" invariant while allowing a shrink it can attribute to reconcile. Legacy fixture: `test_pre697_origin_scan_resumes_legacy_bucket` seeded a destination event the source never had. Previously-imported history is an event the source still has, so the fixture now seeds the older source event; the cursor intent (a re-import would yield 3) is unchanged. README documents the shipped contract and both retained limitations. Git-Session-Id: d15e1888-bd37-4cc5-b7a1-8faf30bf1162
[Medium impact] The PR appears safe to merge; all three previous findings are addressed. SummaryThis PR propagates owner-originated deletions within the seven-day reconciliation window.
All three previous findings are addressed. No new actionable issues were found in the changes since the previous review. Reviews (2) · Last reviewed commit: "fix(aw-sync): reconcile boundary deletio..." · Reviewed by Greptile |
| for (identity, dsts) in dest_by_identity { | ||
| let ts = identity.0; |
There was a problem hiding this comment.
Deleting the newest zero-duration event still leaves its destination copy behind. Its timestamp equals resume, but dest_by_identity only includes rows with timestamps strictly before resume. This loop never sees the deleted row, even when other source events remain inside the window. Repeated syncs keep the same cursor and stale copy until newer data arrives.
Include boundary rows in deletion reconciliation without changing which rows the edit pass owns, and add a zero-duration latest-event test.
There was a problem hiding this comment.
Fixed in 073e273. Destination deletion indexing now includes the resume boundary; source boundary rows still back their live copies, while the edit pass remains strictly before the cursor. The new zero-duration latest-event regression failed before the fix and passes afterward, including live-row preservation and repeated-sync checks.
| // Owner-originated deletions reconciled above are an expected shrink. The | ||
| // invariant this guards is that the *incremental copy* never loses events | ||
| // (#682); a shrink larger than the reconcile reported still fails. | ||
| if new_events_count + (reconciled_deletions as i64) < 0 { |
There was a problem hiding this comment.
Reports show negative new events
Allowing the count to shrink now sends negative values through BucketReport.events_new. Deleting one event with no additions produces “Pulled -1 new events” in summary_message and “imported +-1 events” in the status output. Additions and deletions also cancel each other, hiding work the pass completed.
Track removals separately and keep events_new as the number added, or update the report fields and messages to clearly describe a net change.
There was a problem hiding this comment.
Fixed in 073e273 using the net-change option: the persisted events_new field retains its wire name, but its contract and all summary/status counts now explicitly say net change rather than new events. Pull, push, and both-mode regression checks failed on the old wording and now pass; signed formatting remains intact.
| state | ||
| .ds_dest | ||
| .insert_events(legacy_id, &[before_t0]) | ||
| .unwrap(); | ||
| state.ds_dest.force_commit().unwrap(); |
There was a problem hiding this comment.
Seeding the destination with before_t0 weakens this test's cursor check. The source now contains the exact same row, so boundary_dedup can skip it even if the source fetch starts from the beginning. The final count still reaches two, contrary to the assertion's claim that a full re-import would produce three.
Assert the returned resumed_at value, or check the source fetch's start bound, so this test still catches an ignored cursor.
There was a problem hiding this comment.
Fixed in 073e273. The fixture now asserts the returned resumed_at equals the seeded legacy event end, independently of the final event count. Removed the incorrect claim that dedup necessarily allows a third row when the cursor is ignored. The targeted legacy-resume test and the full aw-sync suite pass.
The reconcile pass can remove rows the owner deleted, so a bucket's events_new is a net delta that can go negative. The summary lines hardcoded a '+' prefix and would print '+-1 events'. Git-Session-Id: d15e1888-bd37-4cc5-b7a1-8faf30bf1162
The desktop pulls from the staging directory, not from the phone, so a deletion that stops at the first hop leaves the event on the third device and a later push can hand the stale copy back. Fails on 10b4b7f. Git-Session-Id: d15e1888-bd37-4cc5-b7a1-8faf30bf1162
🤖 AI code reviewThe PR extends Safe to merge — no P0/P1 findingsConfidence 5/5 ✅ No findings. The diff looks correct to me on this pass. Files changed (6) — the diff as I read it
Previous review passes
Reviewed Maintainer commands
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #855 +/- ##
===========================================
+ Coverage 70.81% 86.00% +15.19%
===========================================
Files 51 90 +39
Lines 2916 13578 +10662
===========================================
+ Hits 2065 11678 +9613
- Misses 851 1900 +1049 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Git-Session-Id: 5f2041dd-74f0-5713-afce-8d815508fd2f
|
@greptileai review |
|
CI-green and mergeable (Greptile 5/5) — waiting only on a maintainer click. This PR is ready to merge, but the bot has pull-only access to this repo and can't self-merge — surfacing it here so it isn't lost. The monitoring loop will stop re-flagging it now that this note is posted. |
Git-Session-Id: 5e8799cc-f71e-5db5-bf84-12512843f86d
Git-Session-Id: 5e8799cc-f71e-5db5-bf84-12512843f86d
Git-Session-Id: 5e8799cc-f71e-5db5-bf84-12512843f86d
|
Pushed the review fixes through
Both timestamp-tie regressions and both final-review regressions were observed failing before their fixes. Final |
Git-Session-Id: 172fcbff-bfec-5486-a51e-4248bebcb802
Git-Session-Id: 172fcbff-bfec-5486-a51e-4248bebcb802
Git-Session-Id: 172fcbff-bfec-5486-a51e-4248bebcb802
|
Pushed fixes through
Both original findings were reproduced as failing tests before their fixes. The alternating-source regression also failed before the final refusal. Final The wrapper's two local reviews are spent; the final commit pushed without another verdict. Current-head CI/review remain required, and this is not a merge-ready claim. No manual thread resolution or extra Greptile trigger: Greptile's older findings were already addressed; these fixes answer the newer AI review. Creation timestamps are a conservative guard, not unique device identity or repair for histories already mixed by older versions. |
Git-Session-Id: 59e2ca4e-813f-565f-be8f-10be5d902275
|
Pushed
Final |
Git-Session-Id: 3ae9de99-8ef0-52eb-888e-fe78b935bb87
|
Pushed
Both reported failures were reproduced before the fix. Final The wrapper reported its existing 2/2 iteration cap and verified the push without another local verdict. Current-head CI has format passing and six checks running; automatic review still needs to verify the fixes and resolve the two threads. This is not a merge-ready claim. No manual thread resolution or Greptile re-trigger; the bounded repair and conservative clipping limitations remain documented. |
What
aw-syncnever propagates an event deletion.aw-sync/README.mdsays so, but the destination copy is not just unsynced — it is permanent: any later replica push copies it back.Repro
aw-sync/tests/owner_deletion.rs(first commit, no production change). Source bucket with three events inside the 7-day window, sync, delete the middle one on the source, sync again:Three propagation tests fail on
10b4b7f2dd: the middle deletion, the newest-event deletion (so it is not just an interior row), and the deletion through the staging hop (phone → staging → desktop, where the desktop pulls from staging rather than the phone).Cause:
reconcile_updated_eventsiteratessrc_by_identity, so it only visits identities that still exist on the source. A deleted event has no identity to visit, and the incremental copy resumes at the destination's newest event end.Approach: absence in the bounded window, not tombstones
aw-syncis single-writer per bucket, and the edit path already treats the source as authoritative for its own bucket ("a same-identity row with different data is an owner-originated edit, not a conflict"). A deletion is the same authority argument with zero surviving rows, so no tombstone record is needed. In-band markers would have to be filtered by the bucket API, WebUI, exports and the Android store; a separate store is a new mechanism with no consumer.After the edit pass, any remaining destination-only identity is removed unless the source still backs it. Two guards keep that from destroying data:
(end, data)fingerprint still exists on the source is a start-clipped copy of a live event — the incremental copy can insert those with a different timestamp and zeroed duration. That fingerprint is the same scheme the resume-boundary dedup already uses; this PR factors it intoboundary_fingerprint()and reuses it at all three sites.reconcile_updated_eventsnow returns the number of removed rows sosync_onecan keep the #682 "event count must not shrink" invariant while tolerating a shrink it can attribute to reconcile.report.rsrenders that net delta with its sign, so a deletion-only bucket no longer prints+-1 events.Retained limitations (now documented)
test_pre697_origin_scan_resumes_legacy_bucketseeded a synthetic destination event the source never had; the fixture now seeds the older source event, which is what "previously-imported history" means. Its cursor intent is unchanged (a re-import would produce 3 events, not 2).Not covered
The reverse case in the issue — a delete that lands on the destination device while its insert fails, leaving the event missing there — needs a repair pass for rows missing below the cursor. That is a different mechanism and is not claimed here.
Verification
cargo test -p aw-sync: 91 + 94 + 9 + 5 + 24 + 3 passed, 0 failed.cargo fmt -p aw-sync -- --checkandcargo clippy -p aw-sync --all-targetsclean.origin/master'saw-sync/src/sync.rswith the final test file; the two contract tests (7-day limitation, empty-window guard) pass before and after.