Skip to content

QVAC-26761 feat[api]: measure ABot world session memory fit - #54

Merged
gianni-cor merged 7 commits into
codex/qvac-26761-diffusion-fitfrom
codex/qvac-26761-abot-fit
Oct 12, 2026
Merged

gianni-cor merged 7 commits into
codex/qvac-26761-diffusion-fitfrom
codex/qvac-26761-abot-fit

Conversation

@DmitryMalishev

@DmitryMalishev DmitryMalishev commented Oct 8, 2026 •

Copy link
Copy Markdown

Summary

ABot uses a separate world-session API, so ordinary diffusion fit does not measure its DiT, taehv decoder or scene pack. Add metadata-only sd_abot_fit_params for the configured placement and a stated walk length, defaulting to 100 steps.

The estimate includes graph arenas, parameter staging, attention caches, scene/frame buffers and retained latent history. Decode sizing follows the largest history overlap reached by the requested session. Disk DiT placement uses measured streaming staging under its graph budget, and Apple arm64 Metal allocations contribute to combined host-memory use. Session structures and generation/history behavior are preserved; the workload length limits only the estimate.

Related Issue / Discussion

Stacked on ESRGAN engine #53. Review this PR's added ABot changes separately; merge #53 first, then retarget/synchronize this branch and validate. Addon integration: QVAC #4939.

Additional Information

  • Current head: e438cd215f96ce678294527def6e7c8b19e25190; ABot review fixes: 3d3c879fe2fc06607d981d59b4948057014971c6.

  • Windows MSVC and Clang/static-CRT builds: 15/15 CTests. Linux CPU, CUDA 13.3/SM120a and Vulkan builds against verified current sources: 15/15 each; GPU execution restricted to one RTX 5090.

  • Tests cover header-only checkpoints, default/explicit lengths, retained-history growth, recompute/KV, controlled low memory, scene metadata, and decoder overlap for 1/2-frame blocks. Added platform-aware mocked-device regressions for shared host/device budgets. The IGPU case runs locally; an additional Windows mock-device simulation executes the exact current helper's Apple arm64 branch for MTL-GPU, non-MTL GPU, IGPU and missing registry. macOS/iOS CI compiles the actual target. Existing macOS ABot generation gates remain unchanged; no physical Apple world-session test was performed.

  • Real ABot Q8/taehv/scene fit passes for the default 100 steps, CPU parameters, recompute/KV, and CUDA/Vulkan disk streaming with a 4 GiB graph budget. CPU/unbudgeted staging retains conservative full sizing.

  • Downstream temporary-overlay fit: 14 tests/90 assertions; full Linux GPU integration: 93/737. ABot fit/load/walk, scene creation, cancellation/reload, CPU/KV and disk/KV parity pass. All 306 addon C++ tests, including the opt-in real Wan case, and current-native SDK worker default/explicit ABot fit checks pass. Wan 2.2 and LTX video/audio opt-ins pass.

  • Automated detail guards pass: luminance deviations 38.2 and 51.6 exceed 20; CPU/KV and disk/KV mean pixel errors are 0.0000 against resident execution. No human visual review, pre-change golden comparison or 100-step generation-quality run was performed.

  • Fit requires an existing scene pack. Scene creation with text/image encoders is outside this session estimate. No dependency versions change.

  • Current engine CI.

  • Shared RAM accounting combines Metal/IGPU allocations with host buffers and CPU weights. Regressions cover resident and retained ESRGAN weights, CPU offload, time-sharing, streaming segment peaks, split placement and VAE tiling. The shared-residency test reproduces the previous planner's false fit and passes with the fix.

  • The addon overlay pins 4c3c8f6d1bc15cdde9e3e69b46561e2f361c195c; subsequent engine commits add tests only, with identical src/, include/ and root CMake content.

Checklist

  • I have read and confirmed this PR follows the contribution guidelines.

Final review validation (2026-10-12): the complete current-head engine and addon workflows have concluded successfully, and all registered PR checks are successful or expected skips. Windows revalidation passes all 14 ESRGAN and 15 ABot CTests. The current native stack previously passed all 59 Linux validation steps, including full temporary-overlay GPU suites (92/703 and 93/737), 305/306 C++ tests, video/audio opt-ins and SDK worker calls. Production sources are unchanged by the latest review clarification.

The latest engine #53 allocation review confirms the existing metadata-only dry run, implemented in c7d665d803884b4ea7acfc7131c75c3f4ee92f0c. It uses ggml_gallocr_reserve_n_size with no allocation, skips execution, and requests no output tensor. A validation-only Windows probe measured 180,589,953,088 bytes of compute for a 4096×4096 graph while observing zero payload-allocation and execution calls; positive controls detected one call each. All 2,500 inspected tensors retained null data/buffer pointers. No production, API, dependency or committed-test changes were needed. The extra Linux allocation-instrumentation run has not been performed yet; existing Linux/current-head CI results above remain valid.

Comment thread src/abot_world.hpp Outdated
Comment thread src/stable-diffusion.cpp Outdated
Comment thread src/core/fit_params.cpp Outdated
Comment thread src/stable-diffusion.cpp
Signed-off-by: Dmitry Malishev <dmitry.malishev@tether.io>
…bot-fit

Signed-off-by: Dmitry Malishev <dmitry.malishev@tether.io>

# Conflicts:
#	tests/test-fit-params.cpp
Signed-off-by: Dmitry Malishev <dmitry.malishev@tether.io>
@gianni-cor
gianni-cor merged commit a31e575 into codex/qvac-26761-diffusion-fit Oct 12, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants