Repository navigation
refactor(infra)!: unify CUDA backend configuration - #20645
Merged
tqchen merged 14 commits intoOct 11, 2026
Merged
Conversation
tqchen
reviewed
Oct 10, 2026
| ffi::String text(code->data(), code->size()); | ||
| auto result = compile(text, config.value()).cast<ffi::Array<ffi::Any>>(); | ||
| source->Set("cuda", text); | ||
| source->Set("cuda.compile_config", result[2].cast<ffi::String>()); |
Member
There was a problem hiding this comment.
source remember the code and backend_config as string
tqchen
reviewed
Oct 10, 2026
| ffi::String fmt_; | ||
| // function information table. | ||
| ffi::Map<ffi::String, FunctionInfo> fmap_; | ||
| // In-memory source map for InspectSource — never serialized. |
Member
There was a problem hiding this comment.
ffi::String backend_config_; // serialized json string of backend config;
tqchen
approved these changes
Oct 11, 2026
tqchen
pushed a commit
that referenced
this pull request
Oct 11, 2026
CUDA backend configuration already travels through Target attributes.
Remove the parallel `backend_config` keyword from `tvm.compile`,
`tvm.tirx.build`, `tvm.relax.build`, and `IketProfiler.compile`;
compilation now resolves its defaults from the supplied Target.
```python
import tvm
from tvm.backend.cuda import BackendConfig
target = tvm.target.Target({
"kind": "cuda",
"arch": "sm_100a",
"backend_config": {
"cuda": BackendConfig(compiler="nvrtc", nvrtc=["--use_fast_math", "--ftz=false"])
},
})
executable = tvm.compile(func, target=target)
```
Each `device_entry(backend_config=...)` keeps its independent overrides,
including architecture and toolchain settings. The existing per-entry
resolution and module grouping continue to apply. Update callers, source
helpers, and documentation to use Target defaults followed by entry
overrides.
Follow-up to merged #20645.
Validation:
- Built successfully on the merged #20645 baseline (`0f861ee568`) in 67
seconds.
- Full TIRx suite: 2,689 passed / 290 skipped on seven B200 GPUs, 165
seconds.
- Backend configuration, launch, and IKET regressions: 131 passed / 2
skipped.
- CUDA codegen and Relax VM build regressions: 415 passed / 4 skipped.
- Updated TIRx-kernels callers: 94 focused tests passed; all eight MQA
registry correctness cases passed. Both workload and strict registry
import gates passed (274 workloads / 93 kernels).
- Changed-file and staged pre-commit checks passed.
The latest upstream commit (`27576ce2d1`) fails the local NCCL-enabled
C++ build in `cuda_ipc_memory.cc` after its object-handle changes. The
checks above therefore use the merged #20645 baseline; this follow-up
changes Python APIs, tests, and documentation only.
tqchen
pushed a commit
that referenced
this pull request
Oct 16, 2026
CUDA compiler settings need to be consistent across build defaults,
target presets, and individual device entries without maintaining a
Python field for every native compiler flag. Introduce a lightweight
per-backend mapping accepted directly by `tvm.compile(...,
backend_config=...)` and `T.device_entry(..., backend_config=...)`.
```python
from tvm.backend.cuda import BackendConfig
cuda_config = BackendConfig(
arch="sm_100a",
compiler="nvrtc",
nvrtc=["--use_fast_math", "--ftz=false"],
)
executable = tvm.compile(func, backend_config={"cuda": cuda_config})
```
- CUDA exposes six optional keys: `arch`, `compiler`, `target_format`,
`nvcc`, `nvrtc`, and `ptxas`. Native argv lists are forwarded to the
selected toolchain. `BackendConfig` is a TypedDict whose constructor
returns a regular dictionary.
- Resolve backend defaults, Target/tag defaults, compile overrides, and
entry overrides before architecture-sensitive lowering. Lists replace
inherited lists; an empty list clears them. Snapshot inputs and use
canonical configuration cache keys.
- Compile entries with different architectures or options in separate
existing CUDA modules. Specialize shared private device helpers before
tile dispatch and dtype legalization so each helper follows its caller's
configuration.
- Keep CUDA runtime metadata to one opaque backend configuration string
appended after the existing serialized module prefix. Remove the
versioned metadata helper and persisted compiler logs. Accept legacy
payloads, avoid invoking a compiler for source-only fallback modules,
and replay saved source independently of an unrelated current Target.
- Export `cuda_host` as ordinary C++ with embedded independently
compiled device binaries. Exported libraries require TVM FFI and CUDA,
with no device compiler required at runtime.
- Replace `CompileConfig`, compiler-policy environment variables, and
decorator-level architecture settings. Obsolete compiler-policy
environment variables are ignored. Update build, Relax, external-kernel,
and IKET consumers and document native compiler options separately from
the runtime schema.
The caller migration is in
[mlc-ai/TIRx-kernels#8](mlc-ai/TIRx-kernels#8).
The earlier CUDA launch configuration PR #20639 is merged.
Validation on the current implementation:
- TVM C++ build and changed-file pre-commit checks passed.
- Full TIRx suite: 2,691 passed / 290 skipped on seven B200 GPUs (160
seconds). Workload import gate (274 workloads / 93 kernels) and strict
registry import gate passed.
- CUDA codegen and Relax VM build regressions: 415 passed / 4 skipped.
- Backend configuration tests: 45 passed, covering native flags and FTZ
behavior, Target/tag defaults, mixed NVCC/NVRTC entries, helper
specialization, concurrent compilation, explicit offline architecture,
source fallback, legacy payloads, fresh-process artifact replay, and
standalone C++ host export.
- TIRx-kernels unit tests: 225 passed, including configuration
forwarding, CPU preparation without CUDA initialization, mutable-input
snapshots, and cache keys across Target defaults.
tqchen
pushed a commit
that referenced
this pull request
Oct 16, 2026
CUDA backend configuration already travels through Target attributes.
Remove the parallel `backend_config` keyword from `tvm.compile`,
`tvm.tirx.build`, `tvm.relax.build`, and `IketProfiler.compile`;
compilation now resolves its defaults from the supplied Target.
```python
import tvm
from tvm.backend.cuda import BackendConfig
target = tvm.target.Target({
"kind": "cuda",
"arch": "sm_100a",
"backend_config": {
"cuda": BackendConfig(compiler="nvrtc", nvrtc=["--use_fast_math", "--ftz=false"])
},
})
executable = tvm.compile(func, target=target)
```
Each `device_entry(backend_config=...)` keeps its independent overrides,
including architecture and toolchain settings. The existing per-entry
resolution and module grouping continue to apply. Update callers, source
helpers, and documentation to use Target defaults followed by entry
overrides.
Follow-up to merged #20645.
Validation:
- Built successfully on the merged #20645 baseline (`0f861ee568`) in 67
seconds.
- Full TIRx suite: 2,689 passed / 290 skipped on seven B200 GPUs, 165
seconds.
- Backend configuration, launch, and IKET regressions: 131 passed / 2
skipped.
- CUDA codegen and Relax VM build regressions: 415 passed / 4 skipped.
- Updated TIRx-kernels callers: 94 focused tests passed; all eight MQA
registry correctness cases passed. Both workload and strict registry
import gates passed (274 workloads / 93 kernels).
- Changed-file and staged pre-commit checks passed.
The latest upstream commit (`27576ce2d1`) fails the local NCCL-enabled
C++ build in `cuda_ipc_memory.cc` after its object-handle changes. The
checks above therefore use the merged #20645 baseline; this follow-up
changes Python APIs, tests, and documentation only.
tqchen
pushed a commit
that referenced
this pull request
Oct 18, 2026
CUDA compiler settings need to be consistent across build defaults,
target presets, and individual device entries without maintaining a
Python field for every native compiler flag. Introduce a lightweight
per-backend mapping accepted directly by `tvm.compile(...,
backend_config=...)` and `T.device_entry(..., backend_config=...)`.
```python
from tvm.backend.cuda import BackendConfig
cuda_config = BackendConfig(
arch="sm_100a",
compiler="nvrtc",
nvrtc=["--use_fast_math", "--ftz=false"],
)
executable = tvm.compile(func, backend_config={"cuda": cuda_config})
```
- CUDA exposes six optional keys: `arch`, `compiler`, `target_format`,
`nvcc`, `nvrtc`, and `ptxas`. Native argv lists are forwarded to the
selected toolchain. `BackendConfig` is a TypedDict whose constructor
returns a regular dictionary.
- Resolve backend defaults, Target/tag defaults, compile overrides, and
entry overrides before architecture-sensitive lowering. Lists replace
inherited lists; an empty list clears them. Snapshot inputs and use
canonical configuration cache keys.
- Compile entries with different architectures or options in separate
existing CUDA modules. Specialize shared private device helpers before
tile dispatch and dtype legalization so each helper follows its caller's
configuration.
- Keep CUDA runtime metadata to one opaque backend configuration string
appended after the existing serialized module prefix. Remove the
versioned metadata helper and persisted compiler logs. Accept legacy
payloads, avoid invoking a compiler for source-only fallback modules,
and replay saved source independently of an unrelated current Target.
- Export `cuda_host` as ordinary C++ with embedded independently
compiled device binaries. Exported libraries require TVM FFI and CUDA,
with no device compiler required at runtime.
- Replace `CompileConfig`, compiler-policy environment variables, and
decorator-level architecture settings. Obsolete compiler-policy
environment variables are ignored. Update build, Relax, external-kernel,
and IKET consumers and document native compiler options separately from
the runtime schema.
The caller migration is in
[mlc-ai/TIRx-kernels#8](mlc-ai/TIRx-kernels#8).
The earlier CUDA launch configuration PR #20639 is merged.
Validation on the current implementation:
- TVM C++ build and changed-file pre-commit checks passed.
- Full TIRx suite: 2,691 passed / 290 skipped on seven B200 GPUs (160
seconds). Workload import gate (274 workloads / 93 kernels) and strict
registry import gate passed.
- CUDA codegen and Relax VM build regressions: 415 passed / 4 skipped.
- Backend configuration tests: 45 passed, covering native flags and FTZ
behavior, Target/tag defaults, mixed NVCC/NVRTC entries, helper
specialization, concurrent compilation, explicit offline architecture,
source fallback, legacy payloads, fresh-process artifact replay, and
standalone C++ host export.
- TIRx-kernels unit tests: 225 passed, including configuration
forwarding, CPU preparation without CUDA initialization, mutable-input
snapshots, and cache keys across Target defaults.
tqchen
pushed a commit
that referenced
this pull request
Oct 18, 2026
CUDA backend configuration already travels through Target attributes.
Remove the parallel `backend_config` keyword from `tvm.compile`,
`tvm.tirx.build`, `tvm.relax.build`, and `IketProfiler.compile`;
compilation now resolves its defaults from the supplied Target.
```python
import tvm
from tvm.backend.cuda import BackendConfig
target = tvm.target.Target({
"kind": "cuda",
"arch": "sm_100a",
"backend_config": {
"cuda": BackendConfig(compiler="nvrtc", nvrtc=["--use_fast_math", "--ftz=false"])
},
})
executable = tvm.compile(func, target=target)
```
Each `device_entry(backend_config=...)` keeps its independent overrides,
including architecture and toolchain settings. The existing per-entry
resolution and module grouping continue to apply. Update callers, source
helpers, and documentation to use Target defaults followed by entry
overrides.
Follow-up to merged #20645.
Validation:
- Built successfully on the merged #20645 baseline (`0f861ee568`) in 67
seconds.
- Full TIRx suite: 2,689 passed / 290 skipped on seven B200 GPUs, 165
seconds.
- Backend configuration, launch, and IKET regressions: 131 passed / 2
skipped.
- CUDA codegen and Relax VM build regressions: 415 passed / 4 skipped.
- Updated TIRx-kernels callers: 94 focused tests passed; all eight MQA
registry correctness cases passed. Both workload and strict registry
import gates passed (274 workloads / 93 kernels).
- Changed-file and staged pre-commit checks passed.
The latest upstream commit (`27576ce2d1`) fails the local NCCL-enabled
C++ build in `cuda_ipc_memory.cc` after its object-handle changes. The
checks above therefore use the merged #20645 baseline; this follow-up
changes Python APIs, tests, and documentation only.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
CUDA compiler settings need to be consistent across build defaults, target presets, and individual device entries without maintaining a Python field for every native compiler flag. Introduce a lightweight per-backend mapping accepted directly by
tvm.compile(..., backend_config=...)andT.device_entry(..., backend_config=...).arch,compiler,target_format,nvcc,nvrtc, andptxas. Native argv lists are forwarded to the selected toolchain.BackendConfigis a TypedDict whose constructor returns a regular dictionary.cuda_hostas ordinary C++ with embedded independently compiled device binaries. Exported libraries require TVM FFI and CUDA, with no device compiler required at runtime.CompileConfig, compiler-policy environment variables, and decorator-level architecture settings. Obsolete compiler-policy environment variables are ignored. Update build, Relax, external-kernel, and IKET consumers and document native compiler options separately from the runtime schema.The caller migration is in mlc-ai/TIRx-kernels#8. The earlier CUDA launch configuration PR #20639 is merged.
Validation on the current implementation: