A successful catalog fetch does not establish that a model can stream or call tools. Live validation is opt-in and requires a running local OpenVidia service with your configured NVIDIA keys. It uses real inference quota: three small requests per selected model. It never changes the dashboard's active model.
Choose explicit API IDs from /v1/models; a Build playground preview alone is
not proof of public API availability. For example:
python -m openvidia.validate_models \
--models nvidia/nemotron-3-ultra-550b-a55b minimaxai/minimax-m3.1-flash \
--timeout 120 --concurrency 1 \
--output /tmp/openvidia-model-validation-20261113.jsonThe validator checks local service identity, an SSE answer ending in [DONE],
a valid tool invocation, and a follow-up answer consuming each tool result.
One or two model cycles can run concurrently. The timeout bounds the entire
stream/tool cycle for each model. Models are tested only when explicitly listed;
there is no automatic whole-catalog sweep.
JSON reports contain the requested model, timestamps, stage outcomes, cycle
latency, safe error categories and static contract-failure reasons. Prompts,
completions, tool arguments, raw error bodies and credentials are not recorded.
Existing report paths are refused to preserve previous results. Interrupted runs
are recorded as interrupted, with unexecuted selections marked not_run.
When NVIDIA supplies a model identifier in the buffered tool responses, the
report includes it and labels the identity matched, different, or
not_reported. Identity evidence is separate from protocol success: inspect a
different result before treating it as validation of the requested model.
Streaming response bodies are never buffered solely to collect identity data.
Exit status is zero only when every selected protocol check passed; failures
return one and a keyboard interruption returns 130. A timeout is evidence of
latency during that test window, not proof that a model is retired. A short
successful check also does not establish unattended availability or performance
on long, real repository tasks. Test those separately with OpenCode sessions and
the duration-controlled python -m openvidia.soak monitor.
Models → Tested tools uses local checks expiring after six hours. Choose
Recent or All, then Check tools for a 60-second maximum cycle through
the configured local inference path. The authenticated route is
POST /api/validate-model with an explicit provider/model ID. Only one such
check runs at a time. Failed checks replace old successes; results persist in
model-validation.json. This action does not run OpenCode or grant its badge.
The separate availability button has a 45-second deadline and only verifies a
nonempty text completion. Neither metadata reachability nor availability alone
qualifies a model for Tested tools. The standalone CLI reports remain separate
from this local dashboard evidence.
Protocol probes and actual OpenCode execution are different checks. For a
process-local custom provider, start a private server: an already-running
background server cannot inherit a new shell's OPENCODE_CONFIG_CONTENT.
For example, using a temporary fixture with a known expected answer:
task_fixture=$(mktemp /tmp/openvidia-read.XXXXXX)
printf 'OPENCODE_TOOL_OK\n' > "$task_fixture"
OPENCODE_CONFIG_CONTENT='{"provider":{"openvidia-check":{"npm":"@ai-sdk/openai-compatible","options":{"baseURL":"http://127.0.0.1:1919/v1","apiKey":"ignored"},"models":{"minimaxai/minimax-m3.1-flash":{"name":"MiniMax validation","tools":true}}}}}' \
opencode --standalone run --archived \
--model openvidia-check/minimaxai/minimax-m3.1-flash --format json \
"Use the read tool on the absolute path $task_fixture and reply only with its contents. Do not modify files."
rm "$task_fixture"Check for a completed read event and the exact expected answer, not just a zero
process exit code. Keep fixture paths absolute: relative tool paths can resolve
to the active workspace. Run private servers sequentially unless their data
stores are verified to be isolated; concurrent servers sharing a SQLite data
store can fail for reasons unrelated to the model. Retain failures separately
from corrected harness runs and redact logs before sharing results.
See the dated results for the protocol and OpenCode checks performed on 13 November 2026, including failed protocol stages.
See also 14 November results for the follow-up dashboard and OpenCode alias checks.