Repository navigation
QVAC-26761 feat[api]: measure ABot world session memory fit - #54
Merged
gianni-cor merged 7 commits intoOct 12, 2026
Merged
Conversation
This was referenced Oct 8, 2026
DmitryMalishev
marked this pull request as ready for review
October 9, 2026 11:02
iancris
reviewed
Oct 9, 2026
iancris
reviewed
Oct 9, 2026
iancris
reviewed
Oct 9, 2026
dev-nid
reviewed
Oct 9, 2026
Signed-off-by: Dmitry Malishev <dmitry.malishev@tether.io>
…bot-fit Signed-off-by: Dmitry Malishev <dmitry.malishev@tether.io> # Conflicts: # tests/test-fit-params.cpp
Signed-off-by: Dmitry Malishev <dmitry.malishev@tether.io>
iancris
approved these changes
Oct 12, 2026
dev-nid
approved these changes
Oct 12, 2026
This was referenced Oct 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ABot uses a separate world-session API, so ordinary diffusion fit does not measure its DiT, taehv decoder or scene pack. Add metadata-only
sd_abot_fit_paramsfor the configured placement and a stated walk length, defaulting to 100 steps.The estimate includes graph arenas, parameter staging, attention caches, scene/frame buffers and retained latent history. Decode sizing follows the largest history overlap reached by the requested session. Disk DiT placement uses measured streaming staging under its graph budget, and Apple arm64 Metal allocations contribute to combined host-memory use. Session structures and generation/history behavior are preserved; the workload length limits only the estimate.
Related Issue / Discussion
Stacked on ESRGAN engine #53. Review this PR's added ABot changes separately; merge #53 first, then retarget/synchronize this branch and validate. Addon integration: QVAC #4939.
Additional Information
Current head:
e438cd215f96ce678294527def6e7c8b19e25190; ABot review fixes:3d3c879fe2fc06607d981d59b4948057014971c6.Windows MSVC and Clang/static-CRT builds: 15/15 CTests. Linux CPU, CUDA 13.3/SM120a and Vulkan builds against verified current sources: 15/15 each; GPU execution restricted to one RTX 5090.
Tests cover header-only checkpoints, default/explicit lengths, retained-history growth, recompute/KV, controlled low memory, scene metadata, and decoder overlap for 1/2-frame blocks. Added platform-aware mocked-device regressions for shared host/device budgets. The IGPU case runs locally; an additional Windows mock-device simulation executes the exact current helper's Apple arm64 branch for MTL-GPU, non-MTL GPU, IGPU and missing registry. macOS/iOS CI compiles the actual target. Existing macOS ABot generation gates remain unchanged; no physical Apple world-session test was performed.
Real ABot Q8/taehv/scene fit passes for the default 100 steps, CPU parameters, recompute/KV, and CUDA/Vulkan disk streaming with a 4 GiB graph budget. CPU/unbudgeted staging retains conservative full sizing.
Downstream temporary-overlay fit: 14 tests/90 assertions; full Linux GPU integration: 93/737. ABot fit/load/walk, scene creation, cancellation/reload, CPU/KV and disk/KV parity pass. All 306 addon C++ tests, including the opt-in real Wan case, and current-native SDK worker default/explicit ABot fit checks pass. Wan 2.2 and LTX video/audio opt-ins pass.
Automated detail guards pass: luminance deviations 38.2 and 51.6 exceed 20; CPU/KV and disk/KV mean pixel errors are 0.0000 against resident execution. No human visual review, pre-change golden comparison or 100-step generation-quality run was performed.
Fit requires an existing scene pack. Scene creation with text/image encoders is outside this session estimate. No dependency versions change.
Current engine CI.
Shared RAM accounting combines Metal/IGPU allocations with host buffers and CPU weights. Regressions cover resident and retained ESRGAN weights, CPU offload, time-sharing, streaming segment peaks, split placement and VAE tiling. The shared-residency test reproduces the previous planner's false fit and passes with the fix.
The addon overlay pins
4c3c8f6d1bc15cdde9e3e69b46561e2f361c195c; subsequent engine commits add tests only, with identicalsrc/,include/and root CMake content.Checklist
Final review validation (2026-10-12): the complete current-head engine and addon workflows have concluded successfully, and all registered PR checks are successful or expected skips. Windows revalidation passes all 14 ESRGAN and 15 ABot CTests. The current native stack previously passed all 59 Linux validation steps, including full temporary-overlay GPU suites (92/703 and 93/737), 305/306 C++ tests, video/audio opt-ins and SDK worker calls. Production sources are unchanged by the latest review clarification.
The latest engine #53 allocation review confirms the existing metadata-only dry run, implemented in
c7d665d803884b4ea7acfc7131c75c3f4ee92f0c. It usesggml_gallocr_reserve_n_sizewith no allocation, skips execution, and requests no output tensor. A validation-only Windows probe measured 180,589,953,088 bytes of compute for a 4096×4096 graph while observing zero payload-allocation and execution calls; positive controls detected one call each. All 2,500 inspected tensors retained null data/buffer pointers. No production, API, dependency or committed-test changes were needed. The extra Linux allocation-instrumentation run has not been performed yet; existing Linux/current-head CI results above remain valid.