Skip to content

feat(cache): treat files and folders renamed into place as outputs - #916

Open
wan9chi wants to merge 2 commits into
feat/cache-listing-ignored-entriesfrom
feat/cache-rename-targets
Open

wan9chi wants to merge 2 commits into
feat/cache-listing-ignored-entriesfrom
feat/cache-rename-targets

Conversation

@wan9chi

@wan9chi wan9chi commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Motivation

Many tools write under a temporary name and rename the result into place:

  • Vitest's fsModuleCache renames each .tmp-* file to its hashed name.
  • Vite's dependency optimizer renames deps_temp_* to deps.
  • Astro writes .astro/*.tmp and renames them.

fspy records nothing for a rename, so those files aren't outputs. A hit doesn't restore them, and since the task reads them back, removing them is a miss.

Changes

  • fspy records a rename destination as a new access kind, RenameTarget, since it may be a whole folder. A hard-link destination, always a file, is recorded as a write. Both are recorded before the call. This covers the Unix preload, the Linux seccomp supervisor and Windows NtSetInformationFile.
  • A rename target that is a file counts as written. One that is a folder stands for every file under it, while the folders themselves don't count.
  • A write record on a path that is a folder after the run, from opening it with write access, is ignored, so it can't pull in the files under it.
  • The files under a renamed folder are archived too.
  • The folder a rename came from, like Vite's deps_temp_*, is gone after the run, so a probe of it was stored as a plain NotFound input with a new random name in every entry. A path that is gone after the run but that the task wrote in now counts as written. It is still fingerprinted as missing, so validation doesn't change.

E2E cases: folder_renamed_into_place (synthetic, vtt fs rename) and vitest_fs_module_cache (Vitest 5 with fsModuleCache, true clean checkout, its today's miss line redacted with #912's redact).

Part of a stack that lets automatic tracking cache tasks which read their own outputs and list the folders holding them, so they also hit on a clean checkout.

Closes #908.

🤖 Generated with Claude Code


Note

Medium Risk
Changes cross-platform fspy hooks and post-run fingerprint/output archiving; incorrect rename handling could cause stale cache hits or missed restores, but scope is filesystem tracking rather than auth or data exfiltration.

Overview
File-access tracking now records rename destinations as RenameTarget and hard-link destinations as writes, on Unix (preload + seccomp), Windows (NtSetInformationFile), with IPC/schema updates and tests. Destinations are logged even when the syscall fails.

Cache behavior treats a renamed file as written output; a renamed folder expands to every file/symlink underneath for archiving and “rebuildable” input logic. Spurious folder write records from opening a directory for write are ignored so reads of that folder still fingerprint correctly.

Adds vtt fs rename, e2e coverage for folder rename-into-place (Vite-style deps), a Vitest fsModuleCache fixture, and a changelog entry: cache hits restore and tolerate missing rename/hard-link outputs on a clean checkout.

Reviewed by Cursor Bugbot for commit a446e8f. Bugbot is set up for automated code reviews on this repo. Configure here.

@wan9chi
wan9chi added this pull request to stack #920 October 30, 2026 03:17
@wan9chi
wan9chi force-pushed the feat/cache-rename-targets branch from 430963c to b139ee7 Compare October 30, 2026 03:19
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

fspy benchmark

linux

dynamic/launch             change  +0.76%  [ -1.70% ..  +3.32%]  overhead   +54.64%
dynamic/access             change  -1.70%  [ -4.43% ..  +2.16%]  overhead   +19.62%
dynamic/access-relative    change  -1.46%  [ -4.00% ..  +0.31%]  overhead   +68.36%
dynamic/access-contended   change  +2.56%  [ -2.16% ..  +6.05%]  overhead   +46.28%
static/launch              change  +2.01%  [ -0.67% ..  +4.44%]  overhead  +194.89%
static/access              change  +0.00%  [ -0.68% ..  +0.57%]  overhead +1429.00%
static/access-relative     change  +0.05%  [ -0.71% ..  +1.22%]  overhead +1945.71%
static/access-contended    change  -0.07%  [ -1.04% ..  +1.17%]  overhead +1869.27%

macos

dynamic/launch             change  +1.13%  [ -4.30% ..  +7.55%]  overhead  +253.46%
dynamic/access             change  -0.74%  [-53.66% .. +15.89%]  overhead    +0.78%
dynamic/access-relative    change  -1.94%  [-13.77% ..  +5.14%]  overhead  +249.47%
dynamic/access-contended   change  +1.06%  [-11.24% .. +18.38%]  overhead    +3.03%

windows

dynamic/launch             change  -0.33%  [ -5.01% ..  +3.96%]  overhead   +25.15%
dynamic/access             change  +0.00%  [ -2.58% ..  +1.11%]  overhead    +1.52%
dynamic/access-relative    change  +0.00%  [ -1.99% ..  +1.09%]  overhead    +1.29%
dynamic/access-contended   change  +0.71%  [ -6.44% ..  +8.32%]  overhead    +2.91%

@wan9chi wan9chi added the ci: full Run the slow CI jobs (macOS x64, musl) on this pull request label Oct 30, 2026
@wan9chi
wan9chi force-pushed the feat/cache-rename-targets branch from b139ee7 to 8a1becb Compare October 30, 2026 04:07
@wan9chi wan9chi added ci: full Run the slow CI jobs (macOS x64, musl) on this pull request and removed ci: full Run the slow CI jobs (macOS x64, musl) on this pull request labels Oct 30, 2026
@wan9chi
wan9chi force-pushed the feat/cache-rename-targets branch from 8a1becb to 6af0c77 Compare October 30, 2026 06:31
…t miss

Two cases for a task that writes under a temporary name, renames it into
place and reads it back. In `folder_renamed_into_place`, the task writes
`tmp/a.js`, renames `tmp` to `deps` and reads `deps/a.js`, through
`vtt fs`, which gets a `rename` operation, like Vite's dependency
optimizer renaming `deps_temp_*` to `deps`. In `vitest_fs_module_cache`,
Vitest 5 with `fsModuleCache` writes each transformed module to
`node_modules/.vitest-cache/.tmp-*`, renames it to a hashed name in the
same folder and reads it back.

Today the renamed paths are plain inputs, since a rename isn't recorded
as a write, so removing them, as on a clean checkout, is a cache miss,
and they aren't in the cached outputs.

The Vitest case removes `node_modules/.vitest-cache` and
`node_modules/.vite/vitest`, not all of `node_modules/.vite`, which
holds the task cache too, and recreates nothing, as on a clean checkout.
Several inputs are then missing, and vt names whichever its parallel
validation finds first, so the next step redacts its miss reason. The
e2e harness redacts SHA-1 hex digests, since Vitest's entry names hash
the module's absolute path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the feat/cache-rename-targets branch from 6af0c77 to a446e8f Compare October 30, 2026 07:08

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a446e8f. Configure here.

add_files_under(&path, workspace_root, &mut written);
} else {
written.insert(path);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Failed folder rename steals existing tree

Medium Severity

Rename destinations are recorded before the syscall, and any path that is a directory after the run is expanded into written. When the rename fails and that directory already existed, every file under it is treated as a rebuildable output even though the task did not place it there.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit a446e8f. Configure here.

A task that writes under a temporary name and renames the result into
place, like Vitest's `fsModuleCache` renaming each `.tmp-*` file to its
hashed name, or Vite's dependency optimizer renaming `deps_temp_*` to
`deps`, left those paths as plain inputs: fspy recorded nothing for a
rename, so they weren't archived, a hit didn't restore them, and
removing them, as on a clean checkout, was a cache miss.

fspy now records the destination of a rename as a rename target, a new
access kind, since it may be a whole folder, and the destination of a
hard link, always a file, as a write. Both are recorded before the call,
so a failed call is recorded too. The source of a rename needs no record.

- Unix preload: `rename`, `renameat`, `link` and `linkat`, plus
  `renameat2` on Linux and `renamex_np` and `renameatx_np` on macOS.
- Linux seccomp supervisor, for static binaries: `renameat`,
  `renameat2` and `linkat`, plus `rename` and `link` on x86_64.
- Windows: `NtSetInformationFile` with `FileRenameInformation(Ex)` or
  `FileLinkInformation(Ex)`, where `MoveFileExW`, `CreateHardLinkW` and
  `SetFileInformationByHandle` end.

`Rebuildable` now takes rename targets next to writes. A rename target
that is a file counts as written; one that is a folder after the run
stands for every file or symlink under it, recursively, while the folder
and the folders in it don't count as written. A write record on a path
that is a folder, from opening it with write access, is ignored, so it
can't pull in the files under it. The archive takes the written files
before output exclusions and `ignoreOutput` filter them, so the files of
a renamed folder are restored by a hit, may be missing on the next run,
and make their folder rebuildable.

The folder a task filled and renamed away, like Vite's `deps_temp_*`,
is gone after the run, so a probe of it was stored as a plain `NotFound`
input with a new name in every entry. A path that is gone after the run
but that the task wrote in now counts as written: the task made or
removed it. Validation doesn't change, since a stored `NotFound` must be
missing either way.

A `.DS_Store` under a renamed folder doesn't count, as in listings.

Written paths aren't stat'd: a written path that is a folder only needs telling
apart where the task read it, so only a read that fingerprints as a folder
is checked; rename targets are, to find the folders to expand.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wan9chi
wan9chi force-pushed the feat/cache-rename-targets branch from a446e8f to 67fd287 Compare November 1, 2026 11:55
@wan9chi wan9chi added the read-write overlap Caching tasks that read and write the same paths label Nov 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci: full Run the slow CI jobs (macOS x64, musl) on this pull request read-write overlap Caching tasks that read and write the same paths

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Automatic output tracking misses files a task renames into place, so a cache hit restores nothing

1 participant