ZIm/crates/language_models
Umesh Yadav b8c1b54f9e
language_models: Fix Mistral tool->user message sequence handling (#31736)
Closes #31491

### Problem
Mistral API enforces strict conversation flow requirements that other
providers don't. Specifically, after a `tool` message, the next message
**must** be from the `assistant` role, not `user`. This causes the
error:
```
"Unexpected role 'user' after role 'tool'"
```
This can also occur in normal conversation flow where mistral doesn't
return the assistant message but that is something which can't be
reproduce reliably.

### Root Cause
When users interrupt an ongoing tool call sequence by sending a new
message, we insert a `user` message directly after a `tool` message,
violating Mistral's protocol.

**Expected Mistral flow:**
```
user → assistant (with tool_calls) → tool (results) → assistant (processes results) → user (next input)
```

**What we were doing:**
```
user → assistant (with tool_calls) → tool (results) → user (interruption) 
```

### Solution
Insert an empty `assistant` message between any `tool` → `user` sequence
in the Mistral provider's request construction. This satisfies Mistral's
API requirements without affecting other providers or requiring UX
changes.

### Testing
To reproduce the original error:
1. Start agent chat with `codestral-latest`
2. Send: "Describe this project using tool call only"
3. Once tool calls begin, send: "stop this"
4. Main branch: API error
5. This fix: Works correctly

Release Notes:

- Fixed Mistral tool calling in some cases
2025-06-06 12:35:22 +03:00
..
src language_models: Fix Mistral tool->user message sequence handling (#31736) 2025-06-06 12:35:22 +03:00
Cargo.toml Add support for OpenRouter as a language model provider (#29496) 2025-06-03 15:59:46 +00:00
LICENSE-GPL Add language_models crate to house language model providers (#20945) 2024-11-20 18:49:34 -05:00