Skip to content

fix(ios): drain autoreleased Metal objects on the offscreen worker - #1

Draft
mickeypause wants to merge 14 commits into
mainfrom
fix/ios-offscreen-autorelease-pool
Draft

mickeypause wants to merge 14 commits into
mainfrom
fix/ios-offscreen-autorelease-pool

Conversation

@mickeypause

@mickeypause mickeypause commented Nov 2, 2026 •

Copy link
Copy Markdown
Owner

Idea behind this PR

I encountered this issue while porting my drawing app from Skia to tgfx. The app lets users draw on a canvas, but repeated offscreen rendering caused a massive increase in RAM usage, even when the rendered content wasn't visible on screen.

The offscreen worker wasn't draining autoreleased temporary objects between render passes. This PR adds an autorelease pool around each Apple offscreen worker pass so those objects are released correctly.


<------- Anything below here was generated by AI. ------->

Upstream status after rebase

The active contribution is a separate PR in the upstream repository, now at 3313b99 on upstream bcf2223 (merged PR margelo#18). It contains five files, +100/−0 lines. This fork PR remains a historical review: because its base is older, its diff now also includes upstream changes brought in by the rebase. The validation below records the earlier tested revision d8b5c58. After rebase, 406 core tests, core lint and typecheck pass; native/device and memory checks have not been rerun against the updated engine.

Problem and fix

Repeated filtered rendering on a retained offscreen surface can grow process memory even when callers dispose every input and snapshot. The process-wide worker runs on a detached C++ thread without a per-pass autorelease pool.

On Apple, this change runs each worker pass inside @autoreleasepool. The weak surface reference is locked inside that scope, so both rendering and destruction of the worker's temporary strong reference happen before the pool drains. The filtered workload renders correctly with either worker; the external iOS Simulator memory gate detects growth with the base worker and passes with the fix.

Changes

Original fix scope at d8b5c58: five files, +100/−2 lines:

  • Route Apple worker passes through the new Objective-C++ helper; Android retains its direct C++ call.
  • Include the helper, Foundation and C++20 Objective-C++ settings in the Apple SceneTest target. CocoaPods already includes ios/**/*.mm.
  • Add a 69-line offscreen workload and exclude it from the default Harness run. Combine the existing duplicate Jest exclusion keys into one array, preserving the documented manual workloads.

There are no public JS API, wire-format, engine-pin, Swift, podspec, host-sampler, runner-config or contribution-guide changes in this PR. The iOS RSS sampler and its tests are separate, at immutable validation commit 9cec9c4.

Regression and reproduction

The workload retains one 512×512 surface for 450 renders, uploads fresh pixels, applies a sigma-4 blur, awaits every snapshot, and disposes input/output handles in finally blocks. Assertions on the first, second and final frames check uploaded colours, a moving rectangle and blur blending across an interior boundary. Bypassing blur fails the boundary assertion.

Configure the existing ios-sim runner for your simulator and install a matching Debug example build, then run:

cd apps/example
bun run harness src/offscreenLeakSoak.harness.tsx --harnessRunner=ios-sim --testPathIgnorePatterns='$^'

This command checks pixels, not memory. For the external RSS gate, use the sampler and documented commands on the separate validation commit. The Android memory checker in this PR is unchanged.

Validation at d8b5c58 — November 2–3, 2026

Offscreen memory controls

Completed pair Base worker: settled RSS growth Fixed worker: settled RSS growth Pixel assertions
November 2 +54.5 MiB — gate fails +2.0 MiB — gate passes Pass on both
November 3, fresh simulator clone +41.5 MiB — gate fails +0.9 MiB — gate passes Pass on both

Each run completes all 450 renders in a fresh process. The PID-aware host sampler reads ps every two seconds, discards warmup, and compares medians of the first and last settled thirds against a 30 MiB threshold. A PID change discards the previous process's samples. Failed assertions or insufficient samples do not produce a passing memory verdict. The zero-blur control fails its pixel assertion and exits 2; base-worker leak detection exits 1, and fixed-worker runs exit 0.

These are arm64 Debug iOS 27 Simulator measurements with Xcode 27.1 and tgfx alpha.12. RSS is process accounting, not total GPU memory, individual Metal-object identification, a physical-device result, or an FPS benchmark. The data supports this bounded workload; it does not establish that every possible leak is fixed.

Physical iPhone Release validation — November 3

An iPhone 15 Pro on iOS 27.0 completed the same 450-render, 512×512 filtered workload in signed arm64 Release builds. The base control replaces only the worker implementation; both builds use identical temporary startup/signing setup and the same public-API fixture. All upload, moving-rectangle and blur-boundary pixel assertions pass on both.

Worker Early physical footprint Late physical footprint Settled growth
Base 143.2 MiB 145.6 MiB +2.4 MiB
Fixed 127.9 MiB 130.3 MiB +2.4 MiB

Instruments Activity Monitor recorded each app PID separately. Values are median physical memory footprint, comparing 15–30 seconds with 45–60 seconds after starting the workload (14–15 samples per window). This is one pair, not simulator RSS or GPU-only memory. The short hardware run does not reproduce the simulator leak and does not establish a hardware memory reduction; the difference in absolute footprint is not a controlled savings claim.

Both Release workers also pass headless hit testing, PNG encoding/save/reload with pixel verification, empty-save-path rejection and normal disposal. An initial validation-only call after Nitro dispose() crashed in the Release binding on both workers; that invalid call was removed before rerunning the valid lifecycle checks. No use-after-dispose rejection guarantee is claimed. The hardware fixture and signing/startup adjustments are not part of the PR.

Functional and build coverage

Check Recorded result
Library unit tests 404 core + 2 camera tests pass
Core/camera typecheck and lint Pass
Code generation and package archives Both packages pass; no generated drift; tgfx archive includes the new helper
Full iOS Harness, same pinned 24-file order Base and fixed repeats each have 191 passed / 8 failed, with the same eight failures
All manual Harness workloads Offscreen and older on-screen soaks complete; on-screen RSS gate passes; cache-purge workload's two iOS tests pass
Host sampler regression tests 3 pass, including PID-change contamination and insufficient post-restart samples
iOS native builds Base-worker and fixed-worker Simulator Debug and physical iPhone arm64 Release builds pass
Android native build :app:assembleDebug passes; APK contains tgfx for arm64-v8a, armeabi-v7a, x86 and x86_64
Android offscreen runtime, emulator Two fresh-process 450-render Vulkan/SwiftShader soaks pass all pixel assertions and the 30 MiB PSS gate; settled PSS growth −10.9 / −21.6 MiB
Android unit-test task Gradle succeeds, but :app:testDebugUnitTest is NO-SOURCE; no Android unit tests execute
Actual macOS C++ targets SceneTest, SnapshotTest and AllocBench compile/link against pinned engine fba0154
Native executable results SnapshotTest and AllocBench exit 0; SceneTest prints ok, then aborts at shutdown on both workers — see below

Every one of the 27 Harness files received an execution attempt. Missing iOS image baselines were automatically created by Harness even with --ci. Those results are execution/smoke evidence, not verified image-regression parity. Generated snapshots were archived outside the source tree, not committed as approved baselines.

Known failures and coverage limits

  • The eight repeatable Harness failures concern handle/shared-value updates, PNG save/reload and Display-P3 conversion. They reproduce with the base worker and with the fix. An earlier fixed run had a ninth failure in film-grain rendering; that case passed in isolation on both workers and in both full repeats. Its intermittent cause remains undiagnosed.
  • SceneTest completes its assertions and prints SceneTest: ok, but the process does not pass: both native workers then abort with a mutex error during shaderc/glslang ShFinalize in MetalGPU destruction. The fixed crash stack places it on the main thread during shutdown; the base worker reproduces the same error and exit 134.
  • Example and benchmark Jest app smoke suites fail importing ESM dependencies; the example's nine logic tests pass. Existing example typecheck/lint, benchmark lint and eight core formatting violations remain. The CI benchmark acceptance command names a directory absent from this checkout. These sources/configurations are unchanged by this PR; the complete repository is not green.
  • Android offscreen follow-up used an Android 36.1 arm64 emulator reporting Goldfish GFXStream / SwiftShader, a software Vulkan renderer. Both 450-render runs passed; native-heap settled growth was +0.9 / −14.4 MiB. The zero-blur Android control fails its boundary assertion (exit 2, no passing memory verdict). These exercise the unchanged Android path; the Apple autorelease-pool branch is not compiled there. Negative growth reflects sampling/GC changes, not guaranteed RAM savings. Android PSS is not directly comparable to the iOS RSS figures, and these guest-process counters do not establish hardware GPU memory behavior.
  • Physical iPhone coverage is limited to the Release offscreen workload and lifecycle checks above, not the full Harness suite or the Tilt Pen app. No Android-phone run, hardware presentation/FPS test, or Android cache-purge meminfo verdict was obtained. No image-baseline parity or general Release performance claim is made.

The iOS measurements use the same local SDK27 UIScene startup adjustment and optional face-detector autolinking override on both workers; neither is included in this PR. For the real C++ build, the exact private engine and required dependencies were fetched; unrelated missing LFS example images were skipped. Xcode 27 required a local macOS deployment target of 14.0. These C++ builds/runs were Debug on an Apple M1 Max. Temporary setup/source changes were restored.

App demonstration: Tilt Pen before / after

Tilt Pen before and after the offscreen autorelease-pool fix

Full MP4 — 12.6 seconds, 1280×1080, 30 fps · Capture details and checksums

Original app recordings from the investigation, aligned to the same scripted tilt sequence on the same iOS 27 Simulator. Both sides run tgfx 0.0.2-alpha.8, Debug builds, after one warmup. The worker autorelease-pool patch is disabled before and enabled after; the separate RN 0.88 view-startup compatibility patch is retained on both.

Across the recorded drawing segment, mean host process RSS is 2,761.4 MiB before and 744.6 MiB after; UI callback cadence is 59.82 / 60.00 fps. The before clip begins with memory already accumulated. These values describe the captured app state, not fresh-launch growth, GPU-only memory, or GPU presentation FPS.

This app video demonstrates the pool fix on alpha.8. The PR ports the fix to current main (alpha.12), verified separately by the 450-render regression above. The MP4, animated preview and manifest are hosted on a dedicated media branch; no binary files were added to this PR's code diff.

This remains a draft against our fork's main for review before proposing it upstream.

@coderabbitai

coderabbitai Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Tip

🎯 Customize your risk assessments

Guide how CodeRabbit assesses merge risk and security architecture risk. Highlight sensitive changes and explain your security boundaries using Merge Risk Instructions and Security Risk Instructions in your review settings. Security architecture risk assessments require the Advanced plan or above.

✅ Automatic approvals, with your rules

Let CodeRabbit approve pull requests once required threads are resolved, pre-merge checks pass, and your risk limits are met. Enable Automatic approvals in review settings, with overrides for teams or file paths.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why have you moved this to one line maybe there was a reson for separate

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Revert changes , don't change anything here

Comment thread scripts/leak-check.mjs Outdated
// node scripts/leak-check.mjs [--package com.tgfxexample] [--threshold-mb 30]
// [--test src/leakSoak.harness.tsx] [--interval-ms 2000]
// [--runner ci-emu]
// iOS Simulator: --ios-simulator <UDID> --runner ios-sim --metro-port 8094

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably don't need to change this

Comment thread scripts/leak-check.mjs Outdated
#!/usr/bin/env node
// Memory-leak gate: runs the leak soak (apps/example/src/leakSoak.harness.tsx)
// on the connected Android device while sampling `adb shell dumpsys meminfo`
// while sampling Android `adb shell dumpsys meminfo` or iOS Simulator `ps`

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably don't need to change this either

Comment thread scripts/leak-check.mjs Outdated
const THRESHOLD_MB = Number(arg('threshold-mb', '30'));
const TEST = arg('test', 'src/leakSoak.harness.tsx');
const INTERVAL_MS = Number(arg('interval-ms', '2000'));
const IOS_SIMULATOR = arg('ios-simulator', null);

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably don't need to change this eaither I feel

Comment thread scripts/leak-check.mjs

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly feel like changes here we don't probably need to do , do we if I am wrong explain

Comment thread CONTRIBUTING.md Outdated

### Offscreen memory reproduction (iOS Simulator)

`src/offscreenLeakSoak.harness.tsx` uses one retained 512×512 surface for 450

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this is not required by the repo guideliens do not add this

SectionTN and others added 2 commits November 3, 2026 12:34
fitRects started the scaleDown destination at the box size and compared
it with the image size, the reverse of react-native-skia's applyBoxFit
and Flutter's BoxFit.scaleDown. An image larger than its box in either
dimension was stretched to the box or drawn past its edges. Start from
the image size and shrink it to the box, keeping the aspect ratio.
riteshshukla04 and others added 5 commits November 3, 2026 21:39
…ught (margelo#18)

* style(cpp): add a clang-format config and format the native sources

The config follows the code as written (Google base, no column limit), so
adopting it is whitespace only. `bun lint-cpp` applies it with the
clang-format version scripts/clang-format.sh pins.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* style: run prettier and drop unused imports

Prettier had drifted on eight files in react-native-tgfx, and the example
and benchmark apps carried unused imports and a dead constant that eslint
reports as errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(android): start the Vulkan backend on drivers that omit promoted extensions

tgfx requested VK_KHR_timeline_semaphore whenever the device reported the
timeline feature, even when the driver did not list the extension. Drivers
on Vulkan 1.2+ may leave it out, as the Android emulator's does, and
vkCreateDevice then failed, so every canvas reported "found no GPU to render
with". The bump only requests the extension when it is listed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci: replace CI and Review with C++ lint, TypeScript lint and harness workflows

- lint-cpp.yml: clang-format at the pinned version, fails on any diff.
- lint-typescript.yml: tsc, eslint and prettier per package, eslint in
  the example and benchmark apps.
- harness-android.yml / harness-ios.yml: every harness suite on a hosted
  emulator (ci-emu, Vulkan) and a simulator (ios-sim, x86_64 because ML
  Kit has no arm64 simulator slice). tgfx comes from the R2 prebuilt, or
  from source when the pinned submodule commit has none. The CI runners
  write fresh baselines each run and upload them as an artifact; the
  committed baselines stay the reference phone's.

The ci-emu runner now defines its AVD, and the duplicate
testPathIgnorePatterns key that let the leak soak into the default suite
is one list.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(imperative): keep block slots writable after a worklet captures them

In dev builds react-native-worklets redefines every configurable key of an
object a worklet captures, on the original, as a getter frozen at capture
time and a setter that only warns. Block slots defined `value` as a plain
accessor, so once a mapper captured them every JS-thread handle.set() was
dropped and pushValues sent the seeded values. The keys are now
non-configurable, which worklets leaves alone.

The shader-uniform baseline is regenerated on the reference phone: the
old one predates tgfx shader changes and differed by 0.2% of pixels on the
band edges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test(3d): sample the frame at whole pixels

The suite reads pixels at width / 2, which is a half pixel on an
odd-sized frame (563 px on the reference phone, 525 on the emulator).
The fractional index landed on the left edge of the next row, so every
centre check read the white background and six tests failed while the
3D rendering itself was right.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(image): open an absolute path as a file instead of decoding it as base64

loadImage and useImage document "a file path" as a source, but a caller's
string with no scheme always went to the base64 decoder, because a JPEG
payload also starts with "/". Loading back what image.save() wrote failed
with "the given base64 payload is not a supported image format". A leading
"/" plus any character outside the base64 alphabet now means a path, so
"/9j/..." payloads still decode as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: regenerate three baselines on the reference phone

- image-fit-modes: the scaleDown cell now matches contain, as margelo#16
  intended; the baseline still showed the old stretched cell.
- pixels-from-pixels: captured at 600x600 on another device; the
  reference phone draws 563x563, so it could never match.
- shader-procedural-rings-dense: 0.2% of pixels moved on the band
  edges against an August baseline, the same tgfx drift as the
  uniform-patch baseline.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* ci(ios): drop ML Kit so the simulator runs natively

GoogleMLKit ships no arm64 simulator slice, so the iOS harness built the
example for x86_64 and ran it under Rosetta. That was slow enough for nine
suites to miss the 1s mount timeout. The face-blur demo was its only user, so
it goes, along with react-native-vision-camera-face-detector.

The harness now builds arm64 only, and gets a 5 GB Node heap: Metro keeps a
graph per test file in the harness process, past the ~2 GB default on the
7 GB macOS runner.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(canvas): keep a canvas's teardown off a recycled view

Fabric recycles the native view, and can hand it to the next canvas before the
old canvas's unmount effect runs. The teardown then committed an empty scene
to that view, once synchronously and once from a scheduled commit, and the new
canvas went blank until something re-encoded it. Slot writes and shared value
writes never re-encode, so on iOS they never showed. The teardown now flushes
muted, as setDirect already does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(ios): give a recycled canvas the colour space it asks for

A Metal window keeps the space it was built in, and a recycled view kept the
window of its last canvas, so a display-p3 canvas on an sRGB view drew sRGB.
The window is rebuilt when the space changes.

The simulator's CAMetalLayer throws on the XR pixel formats, so a P3 canvas
crashed the app there. It gets 8-bit BGRA, which still carries P3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(example): adopt the scene lifecycle

An app built with the iOS 27 SDK that does not adopt UIScene traps at launch
in UIKit. The window moves to a SceneDelegate; the React Native factory stays
on the AppDelegate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Bound Metal temporaries to each worker drain, including final surface cleanup. Keep the Android C++ path unchanged.

450 filtered renders: RSS growth +51.6 MiB before, -3.7 MiB after (30 MiB gate; Debug iOS Simulator).
Discard earlier-process RSS samples after Harness restarts the app.
Assert blur blends an interior boundary; reject a zero-blur control.

Add regression checks for process changes and insufficient samples.
Restore contributor docs, runner configuration, and Android leak tooling.
Preserve iOS measurement tools on validation/ios-offscreen-memory.
Keep the offscreen workload and its combined Jest exclusion list.
@mickeypause
mickeypause force-pushed the fix/ios-offscreen-autorelease-pool branch from d8b5c58 to 3313b99 Compare November 3, 2026 16:34
mickeypause and others added 3 commits November 3, 2026 19:06
Hold the pool in OffscreenContext instead of around the worker loop.
renderToImage, encode and save take the offscreen device on Nitro's
promise threads, which never drain a pool either. Every offscreen GPU
call already runs inside an OffscreenContext, so one RAII member covers
the worker, those calls and surface teardown.

450 iterations on the iOS 27 simulator:
- offscreen leak soak: +2.5 MiB
- renderToImage: +2.6 MiB (was +83.5 MiB)
- encode('png'): flat (was +50.9 MiB)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants