Repository navigation
Conversation
Thinking models served through Chat Completions return their reasoning in `reasoning_content` (DeepSeek) or `reasoning` and `reasoning_details` (OpenRouter, Vercel AI Gateway). The provider dropped it, so a model lost its plan between tool rounds, and DeepSeek, which requires it back whenever a request has tools, failed with a 400. The reasoning is now kept in the transcript as a reasoning entry before the tool calls or response of the message it came with, and sent back on that message as received. The tool loop and replay both build the message from the entry, so the two requests stay the same and prompt caching keeps working. Other providers' reasoning entries are still left out. Fixes huggingface#326 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4WBsc1Jp8fGufe89MhQFB
|
@gtokman Thanks for checking DeepSeek. Chat Completions is shared by a lot of hosts with different rules, so I think we should take your host-specific option. There are reports that Groq rejects Also, #341 adds |
Hosts that serve Chat Completions don't agree on the reasoning fields: Groq rejects `reasoning` on an assistant message and Mistral rejects `reasoning_content`. A reasoning entry now records the host of the API that wrote it, and its fields go back only on requests to that host; a transcript carried to another host replays its messages without them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B4WBsc1Jp8fGufe89MhQFB
|
Done in b3bfcb0: each reasoning entry now records the base URL's host, and its fields go back only on requests to that host. A transcript carried to another host replays its messages without them. There's a test for both cases, and I updated the description. Sounds good on |
Fixes #326. Builds on #325, as you suggested: the reasoning is stored in the transcript, and the round in flight and the replay both build the message from it.
Thinking models served through Chat Completions return their reasoning in
reasoning_content(DeepSeek) orreasoningandreasoning_details(OpenRouter, Vercel AI Gateway). The provider dropped it, so a model lost its plan between tool rounds, and DeepSeek failed outright.What changes
Transcript.Reasoningentry just before the tool calls or response it came with. The fields are stored in its metadata as received (reasoning_detailsas its JSON text), and the reasoning text is its segment, for display.toolCallsMessagesends the fields back on the round's assistant message, in both the streaming and nonstreaming loops. Replay sends them back on the assistant message after the entry: a tool round's, or a response's.reasoning_contentandreasoningpieces are joined, andreasoning_detailsarrays are concatenated in order, as OpenRouter documents.metadata["provider"] == "openai.chatCompletions"). Anthropic's and other providers' stay display history, as before. If Give Ollama's reasoning back to it in each tool round #341 lands first, this can use itsOrigininstead.On your DeepSeek question
DeepSeek needs the reasoning from earlier turns too, not just within the turn. Its thinking mode guide says:
and that otherwise "the API will return a 400 error." Without
tools, it's ignored and not added to the context.So this keeps reasoning for final responses too, not only for tool rounds, and replays it on every later request. That also suits caching: a message sent with its reasoning is sent the same way on every later turn.
Only to the host that wrote it
Hosts that serve Chat Completions don't agree on these fields (Groq rejects
reasoningon an assistant message, Mistral rejectsreasoning_content), so each entry records the base URL's host, and its fields go back only on requests to that host. A transcript carried to another host replays its messages without them.Tests
In
OpenAIToolCallHistoryTests:swift testpasses, apart from the live Ollama tests, which need a local server and fail the same way onmain.🤖 Generated with Claude Code
https://claude.ai/code/session_01B4WBsc1Jp8fGufe89MhQFB