language_models: Fix Mistral tool->user message sequence handling (#31736)
Closes #31491 ### Problem Mistral API enforces strict conversation flow requirements that other providers don't. Specifically, after a `tool` message, the next message **must** be from the `assistant` role, not `user`. This causes the error: ``` "Unexpected role 'user' after role 'tool'" ``` This can also occur in normal conversation flow where mistral doesn't return the assistant message but that is something which can't be reproduce reliably. ### Root Cause When users interrupt an ongoing tool call sequence by sending a new message, we insert a `user` message directly after a `tool` message, violating Mistral's protocol. **Expected Mistral flow:** ``` user → assistant (with tool_calls) → tool (results) → assistant (processes results) → user (next input) ``` **What we were doing:** ``` user → assistant (with tool_calls) → tool (results) → user (interruption) ❌ ``` ### Solution Insert an empty `assistant` message between any `tool` → `user` sequence in the Mistral provider's request construction. This satisfies Mistral's API requirements without affecting other providers or requiring UX changes. ### Testing To reproduce the original error: 1. Start agent chat with `codestral-latest` 2. Send: "Describe this project using tool call only" 3. Once tool calls begin, send: "stop this" 4. Main branch: API error 5. This fix: Works correctly Release Notes: - Fixed Mistral tool calling in some cases
This commit is contained in:
parent
c304e964fe
commit
b8c1b54f9e
1 changed files with 29 additions and 0 deletions
|
@ -444,6 +444,35 @@ pub fn into_mistral(
|
|||
}
|
||||
}
|
||||
|
||||
// The Mistral API requires that tool messages be followed by assistant messages,
|
||||
// not user messages. When we have a tool->user sequence in the conversation,
|
||||
// we need to insert a placeholder assistant message to maintain proper conversation
|
||||
// flow and prevent API errors. This is a Mistral-specific requirement that differs
|
||||
// from other language model APIs.
|
||||
let messages = {
|
||||
let mut fixed_messages = Vec::with_capacity(messages.len());
|
||||
let mut messages_iter = messages.into_iter().peekable();
|
||||
|
||||
while let Some(message) = messages_iter.next() {
|
||||
let is_tool_message = matches!(message, mistral::RequestMessage::Tool { .. });
|
||||
fixed_messages.push(message);
|
||||
|
||||
// Insert assistant message between tool and user messages
|
||||
if is_tool_message {
|
||||
if let Some(next_msg) = messages_iter.peek() {
|
||||
if matches!(next_msg, mistral::RequestMessage::User { .. }) {
|
||||
fixed_messages.push(mistral::RequestMessage::Assistant {
|
||||
content: Some(" ".to_string()),
|
||||
tool_calls: Vec::new(),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fixed_messages
|
||||
};
|
||||
|
||||
mistral::Request {
|
||||
model,
|
||||
messages,
|
||||
|
|
Loading…
Add table
Add a link
Reference in a new issue