[RyanKung]

Summary

Closes #1203 (including consolidated #1205 items 3, 4, 5).

1. Staged, retry- and cancellation-safe CarrierStore::ensure_recovered ([P2] CarrierStore recovery & partial-commit accounting: prevent retry double-charge, reconcile uncertain commits, and normalize partial writes (#1203, #1205) #1203, Pre-1.0 hardening: effect boundaries, async progress, and ownership duplication #1205 item 3):
   • Rebuilds SlotAccounting in a private staging instance (recover_locked) while scanning removals.get_all() and normalizing elements, publishing *index.accounting.lock() = rebuilt together with index.recovered.store(true, Ordering::Release) only after the entire recovery pass completes.
2. Reconciliation of uncertain removal-store commits on error and cancellation (Pre-1.0 hardening: effect boundaries, async progress, and ownership duplication #1205 item 4):
   • Adds RAII guards (UnfinishedRemovalWrite / UnfinishedRemovalDelete). When removals.put / removals.remove errors, reads back removals.get(key) to reconcile index.accounting against the actual stored record; if read-back fails or the future is cancelled mid-await, reverts optimistic deltas and marks index.recovered = false for full reconciliation on the next operation.
3. Dirty cross-store tracking & normalized reads after partial two-stage commits ([P2] CarrierStore recovery & partial-commit accounting: prevent retry double-charge, reconcile uncertain commits, and normalize partial writes (#1203, #1205) #1203, Pre-1.0 hardening: effect boundaries, async progress, and ownership duplication #1205 item 5):
   • Adds DirtyElements + ElementSyncGuard on RemovalIndex so CarrierStore::get and get_all bypass fast glue for dirty keys, performing a full normalizing Entry::join (E_old ⊔ R_new) so tombstoned or register-floored elements are never served without a restart, and opportunistically repairing elements once idle.
   • Eliminates redundant dots/co_dots clone in element_side.

Test Plan

• cargo test -p rings-core --lib dht::storage::tests (93 passed)
• cargo clippy -p rings-core --all-targets -- -D warnings
• cargo check -p rings-core --lib --target wasm32-unknown-unknown --no-default-features --features wasm

[RyanKung]

Reviewed commit: df891cca6b1ef0ce8f857f1c559c969d223300a8.

Verdict: one confirmed P2 finding needs correction.

[P2] Fence detached backend writes before publishing recovered accounting. Dropping UnfinishedRemovalWrite immediately reverses the charge and sets recovered=false. However, native FileStorage uses run_blocking, whose operation survives dropping its awaiter. A subsequent recovery can acquire the file-index lock before that older operation, scan an empty store, and set recovered=true. The older write then commits with no surviving guard to restore its charge or invalidate recovery. The stored removal remains unaccounted, allowing subsequent admissions to exceed the intended share.

Please retain completion/reconciliation ownership beyond caller cancellation, or establish a backend ordering barrier so recovery cannot overtake submitted writes. Setting a dirty flag only when the awaiter drops is insufficient.

Reproduction: a review-only regression, review_recovery_waits_for_a_cancelled_backends_late_commit cancels a put, recovers while its detached backend task is parked, then releases the commit. It fails with held=0, expected=1, although the removal is stored. This uses a controlled backend matching native detached-work ownership, not a direct filesystem scheduling reproduction. A second reviewer independently checked that native lock acquisition permits this ordering.

Other reviewed dimensions:

• Recovery: staging avoids publishing partial charges after failed normalization or interrupted scans.
• Uncertain outcomes: committed-but-error writes/deletes reconcile against read-back; read-back failure triggers recovery.
• CRDT and concurrency: dirty reads perform a full join; normal PeerRing storage reads and writes share storage_transition. No persisted-format or wire-layout change was found.

Validation: all 10 existing dht::storage::tests::test_removal_store tests passed; the added late-commit regression fails. Full workspace/browser testing was not performed.