Issues

• [Bug]: DeepSeek-V4.1-Flash fails at model init on Hopper (SM90): "CSA2 native MXFP8 projections require SM100/SM103"

  NVIDIA/TensorRT-LLM#20511 · Filter by author Smallfu666 opened on Oct 17, 2026

• [Performance] CuTe DSL MLA FP8-KV decode kernel slower at long KV after #19338

  Customized kernels
  NVIDIA/TensorRT-LLM#20510 · Filter by author lucifer1004 opened on Oct 17, 2026

• [Bug]: Kimi K3 / MLA unit tests on SM107: two KDA files skip SM107, a cute_dsl_mla routing case contradicts the perf table, and an indexed-pool decode test compares two code paths bit-exactly

  Testing
  NVIDIA/TensorRT-LLM#20509 · Filter by author lucifer1004 opened on Oct 17, 2026

• [Bug]: FP8 block-scale Linear hits a device-side assert on SM107: DeepGEMM JIT compiles for sm_107f and its CUTLASS disables the TMEM paths

  Customized kernelsPytorch
  NVIDIA/TensorRT-LLM#20508 · Filter by author lucifer1004 opened on Oct 17, 2026

• [Bug]: trtllm-gen FMHA CudaArchDecl.h numbers Sm107a as 4, but the prebuilt libTrtLlmGen archives use 4 for 105a and 5 for 107a

  Customized kernels
  NVIDIA/TensorRT-LLM#20507 · Filter by author lucifer1004 opened on Oct 17, 2026

• [Bug]: MLA generation-only thop.attention calls size the workspace by the context requests' total KV length (OOM on mixed steps with long cached contexts)

  Inference runtimePytorch
  NVIDIA/TensorRT-LLM#20506 · Filter by author lucifer1004 opened on Oct 17, 2026

• [Bug]: Breakable prefill CUDA graphs replay corrupted outputs for some buckets: the shared output buffer is only weakly referenced

  Customized kernels
  NVIDIA/TensorRT-LLM#20504 · Filter by author lucifer1004 opened on Oct 17, 2026
  1

• [Bug]: Nemotron 3.5 Super VL NVFP4 with --tool_parser qwen3_coder emits arguments "" (not "{}") for no-argument tools; echoing the call back in history then fails with HTTP 400 "arguments must be valid JSON"

  NVIDIA/TensorRT-LLM#20503 · Filter by author satishalluri opened on Oct 17, 2026

• [Bug]: NGramPoolManager skips candidate pruning and violates the is_keep_all=False invariant

  bugSpeculative DecodingWindows
  NVIDIA/TensorRT-LLM#20485 · Filter by author kunalsanga opened on Oct 16, 2026

• [Feature]: Skip unused history for single-token beam stop words

  Inference runtime
  NVIDIA/TensorRT-LLM#20460 · Filter by author yifanQi98 opened on Oct 16, 2026

• [Feature]: Structured profiling metadata for distributed inference tracing

  feature requestScale-out
  NVIDIA/TensorRT-LLM#20407 · Filter by author rikundu opened on Oct 15, 2026
  

• [Bug]: FP8-KV MLA generation hangs on SM107 with the grouped trtllm-gen E4M3 kernels when heads per rank do not divide 64 (e.g. 12)

  Customized kernelsModel optimizationSpeculative Decoding
  NVIDIA/TensorRT-LLM#20382 · Filter by author lucifer1004 opened on Oct 15, 2026
  1

• [Performance]: trtllm-gen fmhaReductionKernel walks all KV partials serially per thread; low-batch decode pays a fixed ~6-13 us per call

  General perf
  NVIDIA/TensorRT-LLM#20379 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Performance]: SM107 trtllm-gen MLA decode: the pre-selection cap mMaxNumCtasPerSeqKv = 16 / max(clusterDimX, 2) limits split-KV at low batch

  General perf
  NVIDIA/TensorRT-LLM#20378 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Bug]: trtllm-gen FMHA JIT (NVRTC) fails for MLA generation kernels on SM107: ptxas register allocation (grouped BF16) and "Array.cpp:376 Out-of-bound access" (FP8 KV)

  Customized kernels
  NVIDIA/TensorRT-LLM#20377 · Filter by author lucifer1004 opened on Oct 15, 2026
  1

• [Bug]: Grouped MLA generation cubins are registered as Dense in kernelMetaInfo.h but named and selected as Causal

  Customized kernels
  NVIDIA/TensorRT-LLM#20376 · Filter by author lucifer1004 opened on Oct 15, 2026
  1

• [Bug]: Grouped MLA generation kernels read out of bounds at batch > 1: the MLA generation path never sets cumSeqLensQPtr / cumSeqLensKvPtr

  Customized kernels
  NVIDIA/TensorRT-LLM#20375 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Performance]: DFlash TRTLLM backend: non-causal block attention runs as context FMHA and scales poorly with context length at batch 1

  General perf
  NVIDIA/TensorRT-LLM#20373 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Performance]: Kimi K3 fused attention-residual epilogue is skipped for speculative verify steps (token ceiling defaults to 1)

  Speculative Decoding
  NVIDIA/TensorRT-LLM#20371 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Bug]: A prompt token id outside the vocabulary crashes trtllm-serve (device-side assert in the embedding gather)

  NVIDIA/TensorRT-LLM#20369 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Bug]: trtllm-serve /v1/chat/completions rejects the standard OpenAI field prompt_cache_key with 400

  OpenAI API
  NVIDIA/TensorRT-LLM#20367 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Bug]: Fused DSpark RMSNorm/RoPE is never used on SM107, although its error messages list SM107 as supported

  Customized kernelsPytorch
  NVIDIA/TensorRT-LLM#20365 · Filter by author lucifer1004 opened on Oct 15, 2026

• [Performance]: KVCacheManagerV2 host-tier copies on NVLink-C2C use SM kernels and leave decode persistently slower

  General perfKV-Cache Management
  NVIDIA/TensorRT-LLM#20363 · Filter by author lucifer1004 opened on Oct 15, 2026
  1

• [Feature] Carry the next-token embedding row in the distributed sampling exchange (no separate embedding collective)

  Decoding/SamplingDisaggregated serving
  NVIDIA/TensorRT-LLM#20245 · Filter by author goyalayus opened on Oct 13, 2026

• [New Model]: Add pplx-decider support

  feature requestnew model
  NVIDIA/TensorRT-LLM#20237 · Filter by author anurags25 opened on Oct 13, 2026