Support vitest 5

[GuillaumeLagrange]

TL;DR

@codspeed/vitest-plugin now supports Vitest 5 as well as 3 and 4. The plugin detects which Vitest is installed and hooks into it the right way. Users on Vitest 3/4 see no change.

What changes for users

• vitest peer range is now ^3.2 || ^4 || ^5.

• On Vitest 5, benchmarks are written with Vitest's new bench fixture, and CodSpeed picks them up as-is:

  test("fibo 15", async ({ bench }) => {
    await bench.compare(
      bench("recursive", () => recursiveFibonacci(15)),
      bench("iterative", () => iterativeFibonacci(15)),
    );
  });

  Each bench() in a bench.compare() group is reported as its own benchmark: file.bench.ts::fibo 15::recursive.

• Without CodSpeed, the plugin does nothing and Vitest runs benchmarks normally.

How it works

Vitest 5 removed the benchmark runner the plugin used to extend. Benchmarks now run through a benchmark.provider module instead.

• Vitest 3/4: same custom runner as before, moved under src/legacy/.
• Vitest 5: CodSpeed registers itself as the benchmark provider (src/v5/provider.ts). It runs each benchmark once under instrumentation in simulation mode, or drives tinybench and records walltime results in walltime mode.
• src/vitestBackend.ts picks one of the two from the version exported by vitest/node. Since vitest is a peer dependency, that's the Vitest running the plugin. Code shared by both lives in src/instrument.ts.
• globalSetup now tolerates running twice, since Vitest 5 runs it once per project and clones a separate benchmark project.

Also in this PR

• fix(core): getGitDir now finds the repository root when .git is a file (git worktrees, submodules). Before, every plugin failed with "Could not find a git repository" when run from a worktree. Inside a submodule, benchmark URIs are now relative to the submodule. Also drops the find-up dependency.
• New with-vitest-v4 and with-vitest-v5 examples, so CI covers every supported major.
• The examples get the plugin injected by pnpm (a copy instead of a workspace symlink), so each one resolves its own Vitest like a real install would. pnpm refreshes the copies after every build.
• README shows the Vitest 5 fixture API.
• CLAUDE.md documents how to exercise a plugin in the three CodSpeed modes.

Testing

Ran locally:

• pnpm turbo run build lint typecheck format test test/integ passes.
• with-vitest-v3, with-vitest-v4, with-vitest-v5 and with-typescript-esm examples, plus the plugin's own benches, in fallback, simulation and walltime modes.
• Running an example's vitest with --root from a directory that has a different Vitest installed picks the right integration.

Known limitations

• On Vitest 5, the per-benchmark [CodSpeed] Checked … lines only show with --reporter=verbose.
• The Vitest 5 simulation path doesn't call mongoMeasurement.start/stop around benchmarks, unlike the Vitest 3/4 runner.

Refs COD-2931

[codspeed]

Merging this PR will degrade performance by 17.78%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚠️ 13 benchmarks spent significant time in system calls

System calls cannot be consistently instrumented, so they are not included in the measure, which understates the real cost. Please switch to the Walltime instrument to accurately measure system calls.

Measurement and system calls

• switch 1
• recursive fibo 10
• test_recursive_fibo_10
• switch 1
• recursive fibo 10
• wait 1ms
• wait 1sec
• wait 500ms
• wait 500ms
• switch 1
• …and 3 more

ℹ️ 25 benchmarks have no flame graph

The flamegraph could not be generated because too much time was spent in runtime internals, making the visualization not useful for performance analysis.

• switch 1
• recursive fibo 10
• test_recursive_fibo_10
• switch 1
• recursive fibo 10
• switch 2
• test sync baz 10
• test_recursive_cached_fibo_10
• test_iterative_fibo_10
• test sync baz 100
• …and 15 more

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 5 improved benchmarks
❌ 5 regressed benchmarks
✅ 175 untouched benchmarks
🆕 79 new benchmarks
⏩ 53 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Mode	Benchmark	BASE	HEAD	Efficiency
❌ ⚠️	Simulation	switch 1	244.7 µs	1,322.1 µs	-81.5%
❌ ⚠️	Simulation	recursive fibo 10	299.8 µs	1,373.9 µs	-78.18%
❌ ⚠️	Simulation	test_recursive_fibo_10	288.5 µs	723.7 µs	-60.13%
❌	Simulation	short body 2	104.2 µs	155.2 µs	-32.89%
❌ ⚠️	Simulation	switch 1	244.6 µs	281.1 µs	-12.98%
⚡ ⚠️	Simulation	recursive fibo 10	1,379.6 µs	300.3 µs	×4.6
⚡	Simulation	recursive fibo 15	636.6 µs	351 µs	+81.4%
⚡	Simulation	short body	196.5 µs	155.2 µs	+26.63%
⚡	Simulation	short body	220.3 µs	177 µs	+24.46%
⚡	WallTime	switch 2	96 ns	84 ns	+14.29%
🆕	WallTime	iterative fibo 15	N/A	624 ns	N/A
🆕	WallTime	iterative fibo 20	N/A	648 ns	N/A
🆕	WallTime	recursive fibo 15	N/A	20.4 µs	N/A
🆕	WallTime	recursive fibo 20	N/A	219 µs	N/A
🆕	WallTime	iterative	N/A	588 ns	N/A
🆕	WallTime	recursive	N/A	20.2 µs	N/A
🆕	WallTime	iterative	N/A	600 ns	N/A
🆕	WallTime	recursive	N/A	219 µs	N/A
🆕	WallTime	fibo 10	N/A	2.3 µs	N/A
🆕	WallTime	fibo 15	N/A	20.3 µs	N/A
...	...	...	...	...	...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing cod-2931-prepare-compatibility-with-vitest-5 (4cc5194) with main (f8564bb)}

Footnotes

1. 53 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

[greptile-apps]

Confidence Score: 5/5

[High impact] This looks safe to merge.

Summary

This PR adds Vitest 5 support for the CodSpeed Vitest plugin. The main changes are:

• Version-specific backend selection for Vitest 3/4 and Vitest 5.
• A new Vitest 5 benchmark provider for analysis and walltime modes.
• Legacy Vitest 3/4 runner code moved under src/legacy.
• Updated examples, docs, tests, and package ranges for Vitest 5.
• A core git-root lookup update for .git files in worktrees and submodules.

_{Reviews (3) · Last reviewed commit: "docs(vitest-plugin): show the vitest 5 b..." · Reviewed by Greptile}

[florian-lefebvre]

Hi @GuillaumeLagrange, what's the status of this PR? We would need it for Astro

[GuillaumeLagrange]

Hi @GuillaumeLagrange, what's the status of this PR? We would need it for Astro

Hey! The support for the benchmark provider has been merged and released upstream by vitest, we plan to merge and release this PR early this week! Don't hesitate to continue pinging me to get updates, and sorry about the delay

[JavierGOrdonnez]

Hi @GuillaumeLagrange, what's the status of this PR? I would need it for various projects. Thanks!